Screen Capture in SIlverlight 4.0

July 6, 2010

Recently I have been doing a very sophisticated Silverlight project. In this project there was a need to capture the screen and to save the image in a data base. The known way to capture a screen is to use a WriteableBitmap class instance (See Jeff Prosise blog about this feature that was added in SL 3.0). To capture the screen we use the code from http://stackoverflow.com/questions/1139200/using-fjcore-to-encode-silverlight-writeablebitmap 

The following code is a service that captures the screen image:

 

    public classSnapshotService : ISnapshotService
   
{
        public byte[] Capture()
        {
            var bitmap =  newWriteableBitmap(Application.Current.RootVisual, null);
           
            returnSaveToArray(bitmap);
        }

        private static byte[] SaveToArray(WriteableBitmap bitmap)
        {
            int width = bitmap.PixelWidth;
            int height = bitmap.PixelHeight;

             const int bands = 3;
            var raster = new byte[bands][,];

            //Code From http://stackoverflow.com/questions/1139200/using-fjcore-to-encode-silverlight-writeablebitmap
          
for(int i = 0; i < bands; i++)
            {
                raster[i] = new byte[width, height];
            }

            for(int row = 0; row < height; row++)
            {
                for(int column = 0; column < width; column++)
                {
                    int idx = (width * row) + column;
                    if(idx > 0)
                    {
                        //NOTE: this might fail due to pixels which might be considered as ‘not trusted’ and
                      
// therefore the access to these pixel will be denied
                        // "WriteableBitmap has protected content. Pixel access is not allowed."
                      
int pixel = bitmap.Pixels[idx];

                        raster[0][column, row] = (byte)(pixel >> 16);
                        raster[1][column, row] = (byte)(pixel >> 8);
                        raster[2][column, row] = (byte)(pixel);
                    }
                }
            }
 
            var model = new ColorModel { colorspace = ColorSpace.RGB };
            var img = new FluxJpeg.Core.Image(model, raster);

            //Encode the Image as a JPEG
           
var stream = new MemoryStream();
            var encoder = new JpegEncoder(img, 60, stream);
            encoder.Encode();

            //Back to the start
           
stream.Seek(0, SeekOrigin.Begin);

            //Get teh Bytes and write them to the stream
           
var binaryData = new Byte[stream.Length];
            stream.Read(binaryData, 0, (int)stream.Length);
            return binaryData;
        }

        public BitmapImage Decode(byte[] image)
        {
            if (image == null) return null;
            var b = new BitmapImage();

            using (var stream = new MemoryStream(image))
            {
                b.SetSource(stream);
                return b;
            }

        }
    }

The main problem with this method is that it is not always work. We use Bing map control and Media Elements in our Visual Tree, This causes the line   int pixel = bitmap.Pixels[idx]; to throw Security exception. The exception is there to protect against violating of Digital Rights (DRM) and it is by design. See this link to read more about the problem. First I thought that the problem comes from the fact that the GIS information and the video stream sources come from a site which is not the same as the Silverlight application site. a Cross-Domain policy file should solve this kind of problems. Cross Domain Policy file should come from the site that contains the elements that get rendered on the visual tree. In this case we cannot control the source site and further investigating proved that cross-domain file is useless for this problem. Since the project was a prototype and we ran out of time a radical solution emerged. I decided to use another known Silverlight application that will run on the client and will take a screen capture of the IE tab from the outside. To capture an IE tab, I had to find the IE tab Windows handle, I played with Spy++, understood the relationship between windows under IE, found out that the Window class of Silverlight is “MicrosoftSilverlight” and created the FindFrameWindow method. The rest is just a Win32 BitBlt and WinForm Bitmap/jpeg support:

using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Text;
using System.IO;

namespace G2ScreenCapturer
{
    class Gdi32
    {
        [DllImport("GDI32.dll")]
        public static extern bool BitBlt(IntPtr hdcDest, int nXDest, int nYDest, int nWidth, int nHeight, IntPtr hdcSrc, int nXSrc, int nYSrc, int dwRop);
        [DllImport("GDI32.dll")]
        public static extern IntPtr CreateCompatibleBitmap(IntPtr hdc, int nWidth, int nHeight);
        [DllImport("GDI32.dll")]
        public static extern IntPtr CreateCompatibleDC(IntPtr hdc);
        [DllImport("GDI32.dll")]
        public static extern bool DeleteDC(IntPtr hdc);
        [DllImport("GDI32.dll")]
        public static extern bool DeleteObject(IntPtr hObject);
        [DllImport("GDI32.dll")]
        public static extern int GetDeviceCaps(IntPtr hdc, int nIndex);
        [DllImport("GDI32.dll")]
        public static extern IntPtr SelectObject(IntPtr hdc, IntPtr hgdiobj);
    }

    class User32
    {
        public delegate bool EnumFunc(IntPtr hWnd, uint lParam);

        [DllImport("User32.dll")]
        public static extern IntPtr GetWindowDC(IntPtr hWnd);

        [DllImport("User32.dll")]
        public static extern int ReleaseDC(IntPtr hWnd, IntPtr hDc);

        [DllImport("User32.dll")]
        public static extern bool EnumChildWindows(IntPtr hWndParent, EnumFunc ef, uint lParam);

        [DllImport("User32.dll", CharSet = CharSet.Auto)]
        public static extern int GetClassName(IntPtr hWnd, StringBuilder text, int nMaxCount);

        [DllImport("User32.dll")]
        public static extern IntPtr FindWindow(string lpClassName, string lpWindowName);

    }

    class ScreenCapturer
    {
        public static byte [] CaptureScreen()
        {

            var hWndSrc = FindFrameWindow();
            var hdcSrc = User32.GetWindowDC(hWndSrc);
            var hdcDest = Gdi32.CreateCompatibleDC(hdcSrc);
            var hBitmap = Gdi32.CreateCompatibleBitmap(hdcSrc,
                                                       Gdi32.GetDeviceCaps(hdcSrc, 8), Gdi32.GetDeviceCaps(hdcSrc, 10));
            Gdi32.SelectObject(hdcDest, hBitmap);
            Gdi32.BitBlt(hdcDest, 0, 0, Gdi32.GetDeviceCaps(hdcSrc, 8),
                         Gdi32.GetDeviceCaps(hdcSrc, 10), hdcSrc, 0, 0, 0x00CC0020);
            var result = GetImage(hBitmap);
            Cleanup(hdcSrc, hBitmap, hdcSrc, hdcDest);
            return result;
        }

        private static void Cleanup(IntPtr hWndSrc, IntPtr hBitmap, IntPtr hdcSrc, IntPtr hdcDest)
        {
            User32.ReleaseDC(hWndSrc, hdcSrc);
            Gdi32.DeleteDC(hdcDest);
            Gdi32.DeleteObject(hBitmap);
        }

        private static byte [] GetImage(IntPtr hBitmap)
        {
             var capture =
                new Bitmap(Image.FromHbitmap(hBitmap),
                           Image.FromHbitmap(hBitmap).Width,
                           Image.FromHbitmap(hBitmap).Height);

            var temporaryimageFile = Path.GetTempFileName();
            capture.Save(temporaryimageFile, ImageFormat.Jpeg);

            byte[] result;

            using (var file = File.OpenRead(temporaryimageFile))
            {
                result = new byte[file.Length];
                file.Read(result, 0, (int)file.Length);
            }
            File.Delete(temporaryimageFile);
            return result;
        }

        private static IntPtr FindFrameWindow()
        {
            IntPtr frameHWnd = IntPtr.Zero;

            var ieWnd = User32.FindWindow("IEFrame", "Title - Windows Internet Explorer");

            User32.EnumChildWindows(ieWnd, (hWnd, lp) =>
            {
                var text = new StringBuilder(500);
                User32.GetClassName(hWnd, text, text.Capacity);
                if (text.ToString() == "MicrosoftSilverlight")
                {
                    frameHWnd = hWnd;
                    return false;
                }
                return true;
            }, 0);
            return frameHWnd;
        }
    }
}

But this is not the end of the story. I needed to pass the captured data to the Silverlight application. I decided that my WinForm app will host a WCF Silverlight friendly service. To do so, I had to deal again with the cross-domain policy file. Thanks to WCF Rest support I could have add a cross-domain policy to a self-hosted service:

  [ServiceContract(Namespace = "http://localhost:8086/")]
  public interface ICrossDomainService
  {
      [OperationContract, WebGet(UriTemplate = "/crossdomain.xml", BodyStyle = WebMessageBodyStyle.Bare)]
      Stream GetPolicy();
  }

  [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, Namespace = "http://localhost:8086/")]
  public class CrossDomainService : ICrossDomainService
  {

      #region IPolicyRetriever Members

      [OperationBehavior]
      public Stream GetPolicy()
      {

          const string result = @"<?xml version=""1.0""?>

              <!DOCTYPE cross-domain-policy SYSTEM 

                   ""http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"">

              <cross-domain-policy>

                  <allow-access-from domain=""*"" />

                  <allow-http-request-headers-from domain=""*"" headers=""SOAPAction""/>

              </cross-domain-policy>";

          WebOperationContext.Current.OutgoingResponse.ContentType = "application/xml";

          return new MemoryStream(Encoding.UTF8.GetBytes(result));

      }

      #endregion

  }

 

The Capture Service:

   [ServiceContract(Namespace = "http://localhost:8086/CaptureService/")]
   public interface ICaptureService
   {
       [OperationContract]
       byte[] Capture();

   }

   [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, Namespace = "http://localhost:8086/CaptureService/")]
   [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
   class CaptureService : ICaptureService
   {
       private readonly CaptureForm _form;

       public CaptureService(CaptureForm form)
       {
           _form = form;
       }
       [OperationBehavior]
       public byte[] Capture()
       {
           var capture = ScreenCapturer.CaptureScreen();
           return capture;
       }
   }

 

The self-hosting code:

 

       private void StartServiceHost()
       {
           var host = new ServiceHost(new CaptureService(this), new Uri("http://localhost:8086/CaptureService"));
           var httpBindingElement = new HttpTransportBindingElement();
           var binaryMessageEncodeing = new BinaryMessageEncodingBindingElement
                                            {
                                                MaxReadPoolSize = int.MaxValue,
                                                MaxWritePoolSize = int.MaxValue,
                                                MaxSessionSize = int.MaxValue,
                                                ReaderQuotas =
                                                {
                                                    MaxArrayLength = int.MaxValue,
                                                    MaxBytesPerRead = int.MaxValue,
                                                    MaxDepth = int.MaxValue,
                                                    MaxNameTableCharCount = int.MaxValue,
                                                    MaxStringContentLength = int.MaxValue
                                                }
                                            };

           var elementCollection = new BindingElementCollection
                                       {
                                           binaryMessageEncodeing,
                                           httpBindingElement
                                       };

           var binding = new CustomBinding(elementCollection)
                             {
                                 SendTimeout = TimeSpan.FromMinutes(30),
                                 ReceiveTimeout = TimeSpan.FromMinutes(30),
                                 CloseTimeout = TimeSpan.FromMinutes(30),
                                 OpenTimeout = TimeSpan.FromMinutes(30),
                                 Namespace = "http://localhost:8086/CaptureService"
                             };


          host.AddServiceEndpoint(typeof(ICaptureService), binding, "");

           

           var mex = host.Description.Behaviors.Find<ServiceMetadataBehavior>();
           if (mex == null)
           {
               mex = new ServiceMetadataBehavior {
                   HttpGetEnabled = true
                   
               };
               host.Description.Behaviors.Add(mex);
           }
           var debugBevior = host.Description.Behaviors.Find<ServiceDebugBehavior>();
           if (debugBevior != null)
           {
               debugBevior.IncludeExceptionDetailInFaults = true;    
           }
           else
           {
               host.Description.Behaviors.Add(new ServiceDebugBehavior() {IncludeExceptionDetailInFaults = true});
           }
           

           host.AddServiceEndpoint(
              typeof(IMetadataExchange), binding, "MEX");

           //run as admin or:
           //netsh http add urlacl url=http://+:8086/CaptureService user=…
           host.Open();


           var policyHost = new ServiceHost(new CrossDomainService(), new Uri("http://localhost:8086/"));
           
           var webHttpBinding = new WebHttpBinding
           {
               CrossDomainScriptAccessEnabled = true,
               Namespace = "http://localhost:8086/"
           };

           var policyEndPoint = policyHost.AddServiceEndpoint(typeof(ICrossDomainService), webHttpBinding, "");
           policyEndPoint.Behaviors.Add(new WebHttpBehavior());
           
           //run as admin or:
           //netsh http add urlacl url=http://+:8086/ user=…
           policyHost.Open();
       }

Now, in the Silverlight application I try to capture the screen using the “Correct” SIlverlight WriteableBitmap based method. If I catch an exception, I connect to the capture service and get the image from there. Ugly but works!

 

Last but not least, if you need to use WriteableBitmap for other purposes, take a look at this project: WriteableBitmapEx

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*

15 comments

  1. shairJuly 6, 2010 ב 06:51

    Hi,
    You can call Lock method for WriteableBitmap, the rendering system does not send updates until the WriteableBitmap is fully unlocked by calls to the Unlock method, This should solve the “Pixel access is not allowed.” problem.

    Reply
  2. shairJuly 6, 2010 ב 17:55

    You right!
    I did it through class project and linked to the Silverlight project.

    Reply
  3. NikhilJuly 9, 2010 ב 00:42

    RE: You right!
    I did it through class project and linked to the Silverlight project. by shair

    Can you please help how we can do that ?

    Reply
  4. Pery GroblerSeptember 4, 2010 ב 00:20

    I’m sorry, i didn’t get the part where things get sophisticated… this code reminds me of my 2nd year during junior high school…pfff

    Reply
  5. FurukooOctober 14, 2010 ב 14:01

    Hello

    Can we call unmanaged DLL like winmm.dll in a Silverlight 4 application ?

    Can you provide sample ?

    Thanks a lot

    Reply
  6. Chris WoodhamsDecember 22, 2010 ב 17:14

    Thank you very much for this!

    Reply
  7. amitagyaApril 22, 2012 ב 09:30

    i am also stuck with the same problem of taking snapshot of bing maps. can you please provide me a sample.

    Reply
  8. iconsSeptember 18, 2012 ב 01:14

    I consider, that you are not right. Write to me in PM, we will talk.

    P.S. Please review 24×24 Free Application Icons from Happy Icon Studio

    Reply
  9. icon packageSeptember 18, 2012 ב 11:49

    Excuse, I have removed this question

    P.S. Please review Tab Bar iOS Icons from Iconoman

    Reply
  10. iconsSeptember 18, 2012 ב 12:33

    Excuse, that I can not participate now in discussion – it is very occupied. I will be released – I will necessarily express the opinion on this question.

    P.S. Please review 3D Business Icons from Ikonod

    Reply
  11. qltnfp@gmail.comDecember 12, 2012 ב 04:20

    Are the Israeli leaders getting their civilians, “battle hardened” for what comes next ?Posted by: ana souri | Nov 15, 2012 2:41:20 PM | 15

    Reply
  12. ylmhwckdok@gmail.comJune 18, 2013 ב 07:15

    viewtopic-p-*.html air max 2012 http://ignacnon2013.webnode.fr/ [url=http://ignacnon2013.webnode.fr/]air max 2012[/url]

    Reply