Rendering Video with VMR9 on a Windows Form

6 ביולי 2009

תגיות: , , , ,
4 תגובות

This week I taught a class on video and DirectShow and sat down to prepare some demos.

In this demo I show how to use the Video Mixing Render 9 (VMR9) in Windowless mode to render video on the Form of a Windows Forms application. As in my posts on MSXML I found that the smart pointers of comdef.h and comip.h really helped to clean up the code.

Why, programming the VMR9 in C++ can be as elegant as C# !

You can find the complete source code here.

High Level Design

For this demo I started with the “windowless” sample provided with DirectShow, which is written in C++ against the Win32 native API’s.

With all due respect for the usefullness of the sample, the programming style is difficult to follow. I set out to separate the DirectShow code from the battle field of Win32 native programming and to make the DirectShow code as elegant as C#.

The result is two projects, one a dll written in C++ to handle the DirectShow code and the other a simple Windows Forms application. The Windows Forms application calls the C++ dll with good old Platform Invoke.

I didnt port all the functionality of the “windowless” sample, but that shouldn’t be difficult to do.

Let’s take a closer look at the C++ Library (WindowlessVMR9) and the Windows Forms Application.

The C++ Library: WindowlessVMR9

The API

The API of the library is a set of C-style functions that will be called by the Windows Form application using Platform Invoke. Here they are:

bool VmrInitialize(HWND hwnd);

void VmrCleanUp(HWND hwnd);

bool VmrPlay (LPCWSTR lpcwstrFile);

bool VmrStop ();

void VmrOnPaint (HWND hwnd);

void VmrDisplayModeChanged ();

void VmrMoveVideoWindow(HWND hwnd);

VmrInitialize creates a filter graph manager and a VMR9 renderer filter. It  adds the VMR9 filter to the graph manager and sets its rendering mode to “windowless”. This means that renderer will not create its own window, rather it will use the one it is given by the application.

VmrInitialize then retrieved the IVMRWindowlessControl9 interface from the VMR9 filter and uses it to provide the application window handle to the filter. This function will typically be called when the window loads.

VmrCleanUp releases the resources allocated by VmrInitialize. This function will typically be called when the window closes.

VmrPlay uses the RenderFile method of the filter graph manager’s IGraphBuilder interface to build the graph, based on the VMR9 filter previously added. It then retrieves the IMediaControl interface of the filter graph manager and calls the Run method through it.

VmrStop retrieves the IMediaControl interface of the filter graph manager and calls the Stop method through it.

VmrOnPaint retrieves a device context from the window and passes it to the RepaintVideo method of the IVMRWindowlessControl9 interface of the filter graph manager. This function should be called whenever the window handles a WM_PAINT message.

VmrMoveVideoWindow uses the IVMRWindowlessControl9 interface to reposition the rendered video in the client area of the hosting window’s new coordinates following a WM_SIZE message.

VmrDisplayModeChanged uses the IVMRWindowlessControl9 interface to notify the VMR9 filter that the display mode (e.g. the display resolution or the DPI) has changed. This function will rarely be called.

Smart Pointers Revisited

Before we dive into the implementation of the WindowlessVMR9 library, lets introduce some smart pointers for our DirectShow interfaces.

I started with this:

#include <comdef.h>

#define COM_SMARTPTR_TYPEDEF(Interface) _COM_SMARTPTR_TYPEDEF(Interface, IID_ ## Interface)

My COM_SMARTPTR_TYPEDEF macro exploits the _COM_SMARTPTR_TYPEDEF macro of comdef.h and the COM naming conventions for interfaces and their GUIDS to allow us to define smart pointers based on the _com_ptr_t template like so:

COM_SMARTPTR_TYPEDEF(IGraphBuilder);

and like so

COM_SMARTPTR_TYPEDEF(IMediaControl);

These statements define smart pointers IGraphBuilderPtr and IMediaControlPtr for the IGraphBuilder and IMediaControl interfaces respectively.

Of course, for these to compile you need to have previously included the headers that declare the interfaces themselves, but once done, you can similarly define smart pointers for all the interfaces you need.

Here is the full text of the DirectShowPtr.h include file that I used for this sample:

#pragma once

 

#include <comdef.h>

#include <dshow.h>

#include <d3d9.h>

#include <vmr9.h>

 

#define COM_SMARTPTR_TYPEDEF(Interface) _COM_SMARTPTR_TYPEDEF(Interface, IID_ ## Interface)

 

COM_SMARTPTR_TYPEDEF(IGraphBuilder);

COM_SMARTPTR_TYPEDEF(IFilterGraph);

COM_SMARTPTR_TYPEDEF(IMediaControl);

COM_SMARTPTR_TYPEDEF(IMediaFilter);

COM_SMARTPTR_TYPEDEF(IMediaEvent);

COM_SMARTPTR_TYPEDEF(IEnumFilters);

COM_SMARTPTR_TYPEDEF(IVideoWindow);

COM_SMARTPTR_TYPEDEF(IVMRWindowlessControl9);

COM_SMARTPTR_TYPEDEF(IBaseFilter);

COM_SMARTPTR_TYPEDEF(IVMRFilterConfig9);

Now, to say that _com_ptr_t is a smart pointer is a bit of an understatement.

Apart from having the all the assignment operators, copy constructors and conversion operators you need to make it safe in almost any context, _com_ptr_t also has some very smart COM specific constructors:

Take this statement for example:

        IGraphBuilderPtr graphBuilder (CLSID_FilterGraph);

Here, the constructor of IGraphBuilderPtr will call CoCreateInstance for the co class with the specified CLSID to retrieve a IGraphBuilder interface and store it in graphBuilder. All that in one line!

And look at this one:

        IMediaControlPtr mediaControl(graphBuilder);

Here, the constructor of IMediaControlPtr will call QueryInterface on the graphBuilder for an IMediaControl interface and store it in mediaControl.

With those arrows in our quiver, it would even be safe to write this:

        IMediaControlPtr mediaControl (CLSID_FilterGraph);

        IGraphBuilderPtr graphBuilder (CLSID_FilterGraph);

        mediaControl = IMediaControlPtr(graphBuilder);

and once you get your confidence up, you will even feel comfortable writing this:

        HRESULT hr = IMediaControlPtr(m_GraphBuilder)->Run();

Wow!

_com_ptr_t is defined in comip.h which I included via comdef.h. These files are located in the VC/Include directory of Visual Studio 2008.

Now _com_ptr_t isnt the only smart pointer we could use. ATL also provides smart pointers for COM (ComPtr and ComQIPtr), but I preferred not to include the ATL headers just for these classes.

Also, the Samples\Multimedia\DirectShow\Common folder under the Windows SDK provides a smart pointer class in the SmartPtr.h file, but it isnt half as powerful as _com_ptr_t.

Implementation

Having separated the DirectShow code from the Win32 Window code, and having hidden the complexities of COM behind smart pointers, I think the implementation explains itself:

static IGraphBuilderPtr             m_GraphBuilder;

static IVMRWindowlessControl9Ptr    m_WindowlessControl;

 

bool VmrInitialize(HWND hwnd)

{

    try

    {

        m_GraphBuilder = IGraphBuilderPtr (CLSID_FilterGraph);

 

        IBaseFilterPtr vmrFilter (CLSID_VideoMixingRenderer9);

 

        HRESULT hr = m_GraphBuilder->AddFilter(vmrFilter, L"Video Mixing Renderer 9");

        if (! SUCCEEDED(hr))

            return false;

 

        IVMRFilterConfig9Ptr(vmrFilter)->SetRenderingMode(VMR9Mode_Windowless);

 

        m_WindowlessControl = IVMRWindowlessControl9Ptr(vmrFilter);

        if(! m_WindowlessControl)

            return false;

 

        m_WindowlessControl->SetVideoClippingWindow(hwnd);

    }

    catch (_com_error)

    {

        assert (false);

        return false;

    }

    return true;

}

 

void VmrCleanUp(HWND hwnd)

{

    try

    {

        m_WindowlessControl.Release();

        m_GraphBuilder.Release();

    }

    catch (_com_error)

    {

        assert (false);

    }

}

 

bool VmrPlay (LPCWSTR lpcwstrFile)

{

    if (! m_GraphBuilder)

        return false;

 

    try

    {

        HRESULT hr = m_GraphBuilder->RenderFile(lpcwstrFile, NULL);

        if (! SUCCEEDED(hr))

            return false;

 

        hr = IMediaControlPtr(m_GraphBuilder)->Run();

        return SUCCEEDED (hr);

    }

    catch (_com_error)

    {

        assert (false);

    }

 

    return false;

}

 

bool VmrStop ()

{

    if (! m_GraphBuilder)

        return false;

 

    try

    {

        HRESULT hr = IMediaControlPtr(m_GraphBuilder)->Stop();

        return SUCCEEDED (hr);

    }

    catch (_com_error)

    {

        assert (false);

    }

    return false;

}

 

void VmrOnPaint (HWND hwnd)

{

    if (! m_WindowlessControl)

        return;

 

    PAINTSTRUCT ps;

    HDC         hdc;

    hdc = BeginPaint(hwnd, &ps);

 

    // When using VMR Windowless mode, you must explicitly tell the

    // renderer when to repaint the video in response to WM_PAINT

    // messages.  This is most important when the video is stopped

    // or paused, since the VMR won't be automatically updating the

    // window as the video plays.

 

    try

    {

        HRESULT hr = m_WindowlessControl->RepaintVideo(hwnd, hdc);

    }

    catch (_com_error)

    {

        assert (false);

    }

 

    EndPaint(hwnd, &ps);

}

 

void VmrDisplayModeChanged ()

{

    if (! m_WindowlessControl)

        return;

 

    try

    {

        m_WindowlessControl->DisplayModeChanged();

    }

    catch (_com_error)

    {

        assert (false);

    }

}

 

void VmrMoveVideoWindow(HWND hwnd)

{

    if (! m_WindowlessControl)

        return;

 

    try

    {

        RECT g_rcDest;

 

        GetClientRect(hwnd, &g_rcDest);

        HRESULT hr = m_WindowlessControl->SetVideoPosition(NULL, &g_rcDest);

    }

    catch (_com_error)

    {

        assert (false);

    }

}

I think you will agree that this looks more like C# than native COM coding.

 

The Windows Forms Application

In the Windows Forms application I wrote this Platform Invoke class to enable me to call the WindowlessVMR9 unmanaged code:

    public class Vmr9

    {

        [DllImport("WindowlessVmr9", EntryPoint = "VmrInitialize")]

        public static extern bool Initialize(IntPtr hwnd);

 

        [DllImport("WindowlessVmr9", EntryPoint = "VmrCleanUp")]

        public static extern bool CleanUp();

 

        [DllImport("WindowlessVmr9", EntryPoint = "VmrPlay", CharSet = CharSet.Unicode)]

        public static extern bool Play(string fileName);

 

        [DllImport("WindowlessVmr9", EntryPoint = "VmrStop")]

        public static extern bool Stop();

 

        [DllImport("WindowlessVmr9", EntryPoint = "VmrOnPaint")]

        public static extern void OnPaint(IntPtr hwnd);

 

        [DllImport("WindowlessVmr9", EntryPoint = "VmrDisplayModeChanged")]

        public static extern void DisplayModeChanged();

 

        [DllImport("WindowlessVmr9", EntryPoint = "VmrMoveVideoWindow")]

        public static extern void MoveVideoWindow(IntPtr hwnd);

    }

WinForms allows us to retrieve the native HWND of a Form and Platform Invoke allows us to marshall it to the unmanaged code. The Form code itself has now been reduced to this:

    public partial class Form1 : Form

    {

        public Form1()

        {

            InitializeComponent();

        }

 

        private void Form1_Load(object sender, EventArgs e)

        {

            Vmr9.Initialize(panelVideo.Handle);

 

            buttonPlayStop.Text = "Play";

            buttonPlayStop.Enabled = false;

        }

 

        protected override void OnPaint(PaintEventArgs e)

        {

            base.OnPaint(e);

            Vmr9.OnPaint(panelVideo.Handle);

        }

        protected override void OnResize(EventArgs e)

        {

            base.OnResize(e);

            Vmr9.MoveVideoWindow(panelVideo.Handle);

        }

 

        private void Form1_FormClosing(object sender, FormClosingEventArgs e)

        {

            Vmr9.CleanUp();

        }

 

        private void buttonOpen_Click(object sender, EventArgs e)

        {

            DialogResult result = openFileDialog1.ShowDialog();

            if (result == DialogResult.OK)

            {

                textBox1.Text = openFileDialog1.FileName;

                buttonPlayStop.Enabled = true;

            }

        }

 

        private void buttonPlayStop_Click(object sender, EventArgs e)

        {

            if (buttonPlayStop.Text == "Play")

            {

                buttonPlayStop.Text = "Stop";

                buttonOpen.Enabled = false;

                Vmr9.Play(openFileDialog1.FileName);

                Vmr9.MoveVideoWindow(panelVideo.Handle);

            }

            else

            {

                buttonPlayStop.Text = "Play";

                buttonOpen.Enabled = true;

                Vmr9.Stop();

            }

        }

    }

Summary

In this post I demonstrated how to use the Video Mixing Renderer 9 in a Windows Forms Application in windowless mode.

The code is based on the “windowless” sample of the Windows SDK but has been simplified by separating the VMR9 code into an unmanaged dll, replacing the Win32 GUI with Windows Forms and relying on smart pointers to hide the complexities of COM programming.

הוסף תגובה
facebook linkedin twitter email

כתיבת תגובה

האימייל לא יוצג באתר. שדות החובה מסומנים *

4 תגובות

  1. Bobby Montalvo23 באוקטובר 2009 ב 15:33

    This is a great article, I really appreciate your explanation. However, I cannot compile this solution in Release Mode in .NET 2008. I get a LNK2001: unresolved external symbol error for the CLSID's and the IID's. Could you please explain how I could resolve this issue? It does work in Debug mode btw.

    הגב
  2. Bobby Montalvo23 באוקטובר 2009 ב 15:46

    Hi, I was able to fix the unresolved external symbol errors by adding the dependency to 'strmiids.lib' in the WindowlessVmr9 project. However, now I get this error:

    System.DllNotFoundException was unhandled
    Message="Unable to load DLL 'WindowlessVmr9': The specified module could not be found. (Exception from HRESULT: 0x8007007E)"
    Source="Vmr9InWindowsForm"
    TypeName=""
    StackTrace:
    at Vmr9InWindowsForm.Vmr9.MoveVideoWindow(IntPtr hwnd)
    at Vmr9InWindowsForm.Form1.OnResize(EventArgs e) in C:\Documents and Settings\Owner\My Documents\Programming Projects\VMR9 Examples\Vmr9InWindowsForm\Vmr9InWindowsForm\Form1.cs:line 29
    at System.Windows.Forms.Control.OnSizeChanged(EventArgs e)
    at System.Windows.Forms.Control.UpdateBounds(Int32 x, Int32 y, Int32 width, Int32 height, Int32 clientWidth, Int32 clientHeight)
    at System.Windows.Forms.Control.UpdateBounds(Int32 x, Int32 y, Int32 width, Int32 height)
    at System.Windows.Forms.Control.SetBoundsCore(Int32 x, Int32 y, Int32 width, Int32 height, BoundsSpecified specified)
    at System.Windows.Forms.Form.SetBoundsCore(Int32 x, Int32 y, Int32 width, Int32 height, BoundsSpecified specified)
    at System.Windows.Forms.Control.SetBounds(Int32 x, Int32 y, Int32 width, Int32 height, BoundsSpecified specified)
    at System.Windows.Forms.Control.set_Size(Size value)
    at System.Windows.Forms.Control.SetClientSizeCore(Int32 x, Int32 y)
    at System.Windows.Forms.Form.SetClientSizeCore(Int32 x, Int32 y)
    at System.Windows.Forms.Control.set_ClientSize(Size value)
    at System.Windows.Forms.Form.set_ClientSize(Size value)
    at Vmr9InWindowsForm.Form1.InitializeComponent() in C:\Documents and Settings\Owner\My Documents\Programming Projects\VMR9 Examples\Vmr9InWindowsForm\Vmr9InWindowsForm\Form1.Designer.cs:line 133
    at Vmr9InWindowsForm.Form1..ctor() in C:\Documents and Settings\Owner\My Documents\Programming Projects\VMR9 Examples\Vmr9InWindowsForm\Vmr9InWindowsForm\Form1.cs:line 10
    at Vmr9InWindowsForm.Program.Main() in C:\Documents and Settings\Owner\My Documents\Programming Projects\VMR9 Examples\Vmr9InWindowsForm\Vmr9InWindowsForm\Program.cs:line 18
    at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
    at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
    at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
    at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
    at System.Threading.ThreadHelper.ThreadStart()
    InnerException:

    Any help would be greatly appreciated

    הגב
  3. David Sackstein24 באוקטובר 2009 ב 22:15

    Hi Bobby,
    Please check that the project that creates 'WindowlessVmr9.dll' is configured to the same output directory as the WinForm application.
    I did this in the debug configuration, but may have not done so in the release configuration.
    David

    הגב
  4. Learner9925 בינואר 2011 ב 17:39

    Excellent article. I have implemented something similar using DShow.Net, and I'm having flicker issues when resizing the form. I notice your implementation does too. Have you found a solution? Thanks

    הגב