Essential WPF

With the release of Windows Vista and WPF, Microsoft has also released the WIC: "an extensible framework for working with images and image metadata. WIC makes it possible for independent software vendors (ISVs) and independent hardware vendors (IHVs) to develop their own image codecs and get the same platform support as standard image formats (for example, TIFF, JPEG, PNG, GIF, BMP, and WMPhoto)" (MSDN). Unsurprisingly, WPF uses WIC in its core (for example, BitmapSource). Unfortunately, the WPF team chooses to keep the WIC low level API internally, in such a way that there isn't a simple solution for dealing with imaging without consuming a lot of memory and CPU time.

About few months ago, my customer asked me to develop an image processing control based on WPF for a medical-system. This includes several post-processing operations such as zooming, panning, windowing (for B/W) and other implemented on the server side. So I started to investigate the WPF imaging system. First I tried to load raw B/W images (1~18Mb), using the BitmapSource.Create method and the Image visual type. This method worked fine until it came to image-processing. Think that you have a 1Mb-pixel image, and you have to iterate through all of its pixels (1048576) on every mouse movement, calculating the next pixel value, and update the screen. With an Intel Duo Core2 CPU the calculation part is insignificant, but what about updating the screen? Now that I have a new buffer, how do I update the BitmapSource instance?

I was thinking about "disposing" the old BitmapSource image, and then create a new one using the Create method again, since it consumes a lot of memory. Alas! The BitmapSource type is not disposable! Also, there is no Close method. So I putted a null in the BitmapSource reference (just for fun), and created a new BitmapSource. Since the image size is 1M, The CLR considering it as a large object. Large objects are kept in the large-object-heap for a long term (calling the GC.Collect method is not an option!).

The best way of course is to manipulate the already allocated BitmapSource internal buffer. But let put it that way: WPF has no public methods for manipulating the BitmapSource internal buffer (which is WIC of course).

Digging inside BitmapSource with Reflector, I figured out that BitmapSource calls WIC native API's for handling its buffer internally. As I already said, the WPF team did not expose these methods.

Based on this knowledge, I have developed a .NET component for manipulating the internal buffer, by accessing these methods using the .NET great tool ever, reflection.

Before I will show you my code, I want to forward you to Jeremiah Morrill post. He did exactly the same, but posted it three months before me. He owns all the credit for its great job.

My solution is composed of three types: WICBitmap, WICBitmapBuffer and WICBitmapLock.

WICBitmap - encapsulates a BitmapSource instance by providing operations for locking (accessing) the internal WIC image buffer.

WICBitmapLock - represents a WIC buffer lock.

WICBitmapBuffer - encapsulates a WIC buffer pointer and its size.

The code snippet bellow demonstrates how to create a BitmapSource image, and how to access its internal buffer using the WICBitmap* types.

private void InitializeImage()
{
    Uri imageUri = new Uri(@"Images\View.jpg", UriKind.Relative);
    BitmapSource bitmapSource = new BitmapImage(imageUri);
    _imageVisual.Source = bitmapSource;         // WPF Image type
    _wicBitmap = new WICBitmap(bitmapSource);   // WICBitmap type
}

void button_Manipulate(object sender, RoutedEventArgs e)
{
    try
    {
        using (WICBitmapLock bitmapLock = _wicBitmap.Lock())
        {
            WICBitmapBuffer bitmapBuffer = bitmapLock.Data;
            unsafe
            {
                Int32* pStart = (Int32*)bitmapBuffer.Buffer.ToPointer();
                Int32* pEnd = pStart + bitmapBuffer.Size / sizeof(Int32);
                for (; pStart != pEnd; ++pStart)
                {
                    //           RRGGBB
                    *pStart &= 0x0000FF;
                }
            }
        }
        _imageVisual.InvalidateVisual();
    }
    catch (Exception ex)
    {
        MessageBox.Show("Failed to manipulate image:\n" + ex.Message);
    }
}

As you can see, the code above is very simple so there is no need for explanation.

There are some disadvantages for using this mechanism:

1. You should call Image.InvalidateVisual() explicitly after manipulating the buffer (this informs the Image that the BitmapSource data was changed and that it needs to render the image again).
2. Sometimes the Lock/Copy fail. I think that it because I'm trying to lock the buffer while it is in use by the render thread (use the TryLock method instead of Lock, and try again and again if fails. The second or third lock should succeed).
3. You have to copy at least the first image with BitmapSource. I didn't find an option to wrap the first/original buffer like in GDI+ (if you have a lot of images, I suggest to create a pool of BitmapSource instances and reuse them).
4. This solution is temporary and may not work in the future versions of WPF (it works on 3.5), since it uses internal methods. Use it on your own risk.

NOTE: There is another half documented solution for manipulating an image: Custom Bitmap Effects. Creating a custom bitmap effect is involved with C++/COM programming. Maybe I will have time to demonstrate this in a future post.

Now that you have an access to the BitmapSource WIC buffer, you don't need to create BitmapSource instances for each post-frame, which consumes a lot of memory and CPU time.

Download the code (Orcas) from here.

In the next post I will explain how the WIC interop code works, and how to create an image stream (Video) based on my solution.