Overview
.NET is great platform, it speeds up the development process, you deal with your application logic and in most cases you don't need to know that there is whole Windows operating system down there. However sometimes you do need to program against Windows without the .NET assistance. As a Windows developer, you need to keep all your weapons ready, be it .NET, COM or C++ with the native Win32 API. In this article I am going to show how to deal with Windows Message based communication protocol within .NET application.
Windows Messages
In traditional User32.dll based Windows application the Windows Messages mechanism serves as a communication channel among many parties. The Windows Operating System uses this mechanism to ask the Window to draw itself, to inform about environmental change or even to communicate a logoff event. The application itself uses this mechanism to create an event driven design model with a single thread or to communicate between threads. Other applications can use this mechanism with an application defined message (the WM_APP message range) to trigger a logical event, or even to pass memory buffer using the WM_COPYDATA.
Windows In Windows Are Based On An Object Oriented Mechanism
One of the impressive thing about the windows mechanism of Windows, considering its age and the fact that it was implemented with the C language in mind, is that is based on the Object Oriented paradigm. Each window is belong to a Window Class. The class is the factory of a window. The class defines the window's properties as well as the window's behavior. The properties are set using one of the predefined window class member such as the window's menu, or cursor. The behavior is defined by specifying the windows procedure address. The window procedure handles the incoming Windows Messages, which are in fact the methods of the window. Changing the Windows attributes can be made using the SetWindowLong/Ptr. Creating type fields (static) and instance fields can be made by allocating user-defined memory area for each Window class (static) and Window instance (dynamic) using the cbClsExtra and cbWndExtra fields of the WNDCLASS structure. Subclassing a Windows class can be achieved by changing the window procedure address to point to a custom function that handles the Windows Messages before it calls the original window procedure. You can subclass a Window instance or Window class (global subclassing).
Windows Do Not Run
In first look, it seems that that message communication is based on one window sending messages to another window, however windows do not run, threads do. Windows makes a correlation between threads and windows. The thread that calls CreateWindow/Ex is the thread that receives the Windows Messages related to this Window. In a common implementation of a Windows application, the main thread creates it's main and child windows, and then starts a message loop, also known as message pump, pumping messages that has been sent to any of it's windows, and dispatching this messages to the correct Window Procedure – the code that handle Windows Messages on behalf of the target Window.
Using Windows Messages Not For Painting The Window
Passing Windows Messages is one of the most easier inter process communication (IPC) mechanism, especially if the two parties are already implementing the Windows message handling boilerplate to handle their UI. One of the compelling feature of the mechanism is that it serves as a base for thread function call dispatching, enables a cross thread context call. For example in COM Single Threaded Apartment (STA), COM uses Windows Messages to dispatch function calls to the thread that created the component, much the same as dispatching messages to the windows created by the thread. This ensures that all the code of the COM component will be executed in a single thread- the idea behind the STA model.
Message Only Window
There are cases that you need to communicate using Windows Messages with another application or component, but your application is not based on the traditional user32.dll Windows. This happens more and more in modern native and manage applications that are based on Console or DirectX, for example a .NET WPF based application. In such cases there is no need to create a window to handle these messages, since we do not care about painting the window or handling resize messages, we do not want to pay the cost of such a mechanism and we do not want to handle the hiding of such a window. For that in Windows 2000 Microsoft has added a new type of window type, the message-only-window. Creating such a window is very easy, all you need to do is to provide the HWND_MESSAGE as the window's parent. You can do that in the CreateWindow/Ex or by using the SetParent function. You can use any predefined Window class, for example "STATIC", or "Message", as long as you subclass the window procedure to handle the coming Windows Messages.
Building A Message-Only-Window Client For .NET Applications
For handling system event, the .NET framework provides the Microsoft.Win32.SystemEvents class. However this class is made to receive Windows system event messages and can't be extended to receive custom user messages. You can always use Windows Form window to handle windows messages, however you will need to add reference to System.Windows.Forms.dll even if you don't use Windows Forms, and you will need to hide the form. A second approach, which I took, is to use C++/CLI assembly to create a message-only-window proxy. In this mechanism a native code handles the windows messages dispatching boilerplate. All is left for the managed code is to register a callback delegate on the message received event and ask the mechanism to start the message loop in the context of the thread that should handle the coming windows messages.
The Design
The NativeMessageOnlyWindow is a native class that handles the windows messages receiving, dispatching and handling. The MessageOnlyWindow is a managed C++/CLI wrapper (adapter) class that serves as the .NET interface for the mechanism. To bridge between the two, the MessageOnlyWindow class instantiates the native instance of the NativeMessageOnlyWindow and keeps it as a member that will be deleted on dispose. The managed MessageOnlyWindow class implements the IMessageCallback interface. The native NativeMessageOnlyWindow gets and holds a gcroot<IMessageCallback ^> to the MessageOnlyWindow managed class in the constructor. When the NativeMessageOnlyWindow receives Windows messages it calls to the OnMessageReceivedCallback method of the IMessageCallback interface. To get a better understanding on calling from native to .NET see this blog post.
To start pumping messages, the .NET client calls the MessageOblyWindow instance StartMessageLoop method. The call has to be made by the thread that created the Window that need to be handled. This call will not return until a call to StopMessageLoop will be made from any arbitrary thread.
The MessagePump.h header file
1:
2:
3: // MessagePump.h
4:
5: #pragma once
6: #include <Windows.h>
7: #include <hash_map>
8: #include <vcclr.h>
9:
10: using namespace System;
11: using namespace std;
12:
13: namespace MessagePump
14: { 15: //Callback wrapper
16: public interface class IMessageCallback
17: { 18: //get called each time a new message is arrived to the message only window
19: IntPtr OnMessageReceivedCallback(IntPtr hWnd, UINT message, WPARAM wParam, LPARAM lParam);
20: };
21:
22: //Serves as a Windows Message Loop manager for the message only window
23: class NativeMessageOnlyWindow
24: { 25: private:
26: HWND m_hWnd;
27: bool m_bStop;
28:
29: //This map helps in correlating the hWnd we get in the WndProc to the
30: //NativeMessageOnlyWindow instance of that Window
31: static hash_map<HWND, NativeMessageOnlyWindow *> s_windowsMap;
32:
33: //Enables calling back to .NET interface method
34: gcroot<IMessageCallback ^> _messageCallback;
35:
36: //To stop the message loop, we need to call PostQuitMessage in the context of the
37: //message loop thread. To do that we are sending this message
38: static const unsigned int WM_APPSTOP = WM_APP + 242;
39:
40: //Create the message only window
41: void Create();
42:
43: //The WndProc for the message only window
44: static LRESULT CALLBACK WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam);
45:
46:
47: public:
48: NativeMessageOnlyWindow(const gcroot<IMessageCallback ^> &&callback) : m_hWnd(0),
49: m_bStop(false), _messageCallback(callback) {} 50:
51: //Stop the message loop, the actual stop happens asynchronously, since
52: //we use PostQuitMessage
53: void StopMessageLoop();
54:
55: //A blocking call, the message loop is in the context of the calling thread
56: void StartMessageLoop();
57:
58: //Return the message only window handle
59: HWND GetWindowHandle() const { return m_hWnd; } 60:
61: //Stop the message loop and destroy the instance
62: ~NativeMessageOnlyWindow();
63: };
64:
65: ///<summary>
66: /// Represents a Windows Message that encapsulate in .NET type the WndProc arguments
67: ///</summary>
68: public value class WindowsMessage
69: { 70: private:
71: IntPtr m_hwnd;
72: unsigned int m_uMessage;
73: UIntPtr m_wParam;
74: IntPtr m_lParam;
75:
76: public:
77: /// <summary>
78: /// Construct a new WindowsMessage class that holds the WndProc arguments
79: /// </summary>
80: /// <param name="hWnd">The target windows handle</param>
81: /// <param name="message">The message number</param>
82: /// <param name="wParam">the wParam</param>
83: /// <param name="lParam">the lParam</param>
84: WindowsMessage(IntPtr hWnd, UINT message, WPARAM wParam, LPARAM lParam) :
85: m_hwnd(hWnd), m_uMessage(message), m_wParam(UIntPtr(wParam)), m_lParam(IntPtr(lParam)) {} 86:
87: property IntPtr HWnd { IntPtr get() { return m_hwnd; }} 88: property unsigned int Message { unsigned int get() { return m_uMessage; }} 89: property UIntPtr WParam { UIntPtr get() { return m_wParam; }} 90: property IntPtr LParam { IntPtr get() { return m_lParam; }} 91: };
92:
93: ///<summary>
94: ///Use this class to create a message only window
95: ///</summary>
96: ///<remarks>
97: ///With this class you can receive Windows Messages without the need to use a WinForm instance.
98: ///Message only windows are lightweight mechanisms that serves only for receiving windows messages.
99: ///</remarks>
100: public ref class MessageOnlyWindow : public IMessageCallback
101: { 102: private:
103: NativeMessageOnlyWindow *m_pNativeMessageOnlyWindow;
104:
105: protected:
106: /// <summary>
107: /// This callback functions receives the raw WndProc arguments for any message that has been
108: /// sent to the message-only-window
109: /// </summary>
110: /// <param name="hWnd">The target windows handle</param>
111: /// <param name="message">The message number</param>
112: /// <param name="wParam">the wParam</param>
113: /// <param name="lParam">the lParam</param>
114: /// <returns>The method should return a value for the DefWndProc, usually <see cref="IntPtr::Zero"/>
115: /// is suitable</returns>
116: /// <remarks>When overriding this method, call the base class implementation to fire the
117: /// <see cref="MessageReceived"/> event.</remarks>
118: virtual IntPtr OnMessageReceivedCallback(IntPtr hWnd, UINT message, WPARAM wParam, LPARAM lParam)
119: = IMessageCallback::OnMessageReceivedCallback;
120:
121: public:
122: ///<summary>
123: ///Constructor for the message only window instance
124: ///</summary>
125: ///<remarks>
126: ///With this instance you can receive Windows Messages without the need to use a WinForm instance.
127: ///Register a callback on the <see cref="MessageReceived"/> event and call the
128: ///<see cref="StartMessageLoop"/> to start receiving windows messages.
129: /// Call the <see cref="StopMessageLoop"/> to stop receiving windows messages.
130: /// Dispose the instance whenever you do not need it anymore
131: ///</remarks>
132: MessageOnlyWindow();
133:
134: /// <summary>
135: /// Fires an event that represents a single windows message that has been sent to
136: /// the message-only-window
137: /// </summary>
138: event Func<WindowsMessage, IntPtr> ^MessageReceived;
139:
140: /// <summary>
141: /// Ask the message pump thread to stop.
142: /// </summary>
143: /// <remarks>
144: /// The message pump thread is the thread that called the <see cref="StartMessageLoop"/>.
145: /// The stop operations is asynchronously, This call trigger a WM_QUIT message that will
146: /// eventually stop the loop
147: /// </remarks>
148: void StopMessageLoop() { m_pNativeMessageOnlyWindow->StopMessageLoop(); } 149:
150: /// <summary>
151: /// Start the message loop in the context of the calling thread
152: /// </summary>
153: /// <remarks>
154: /// Call this method in the context of the thread that triggered the Windows Messages source.
155: /// This is a blocking call, the message loop is in the context of the calling thread.
156: /// This method will be return only after a call to the <see cref="StopMessageLoop"/>.
157: /// </remarks>
158: void StartMessageLoop() { m_pNativeMessageOnlyWindow->StartMessageLoop(); } 159:
160: /// <summary>
161: /// The message-only-window handle
162: /// </summary>
163: property IntPtr WindowHandle { IntPtr get() 164: { return IntPtr(m_pNativeMessageOnlyWindow->GetWindowHandle()); }} 165:
166: /// <summary>
167: /// clear native resources
168: /// </summary>
169: ~MessageOnlyWindow() { delete m_pNativeMessageOnlyWindow; } 170: };
171: }
172:
The MessagePump.cpp code file
1:
2: //
3: // Messagepump.cpp
4:
5: #include "stdafx.h"
6:
7: #include "MessagePump.h"
8:
9: using namespace System;
10: using namespace System::Runtime::InteropServices;
11:
12:
13: namespace MessagePump
14: { 15: MessageOnlyWindow::MessageOnlyWindow()
16: { 17: m_pNativeMessageOnlyWindow = new NativeMessageOnlyWindow(gcroot<IMessageCallback ^>(this));
18: }
19:
20: IntPtr MessageOnlyWindow::OnMessageReceivedCallback(IntPtr hWnd, UINT message, WPARAM wParam, LPARAM lParam)
21: { 22: return MessageReceived(WindowsMessage(hWnd, message, wParam, lParam));
23: }
24:
25: hash_map<HWND, NativeMessageOnlyWindow *> NativeMessageOnlyWindow::s_windowsMap;
26:
27: void NativeMessageOnlyWindow::Create()
28: { 29: //Create a Window that can only receive messages
30: m_hWnd = CreateWindow(L"Message", L"", 0, 0, 0, 0, 0, HWND_MESSAGE , nullptr, GetModuleHandle(0), nullptr);
31: if (m_hWnd == nullptr)
32: Marshal::ThrowExceptionForHR(GetLastError());
33:
34: //Set the Window's WndProc to our Windows Procedure
35: auto result = SetWindowLongPtr(m_hWnd, GWLP_WNDPROC, (LONG_PTR)WndProc);
36: if (result == 0)
37: Marshal::ThrowExceptionForHR(GetLastError());
38:
39: //Keep the correlation between the hWnd and the MessageOnlyWindow instance
40: s_windowsMap[m_hWnd] = this;
41: }
42:
43: void NativeMessageOnlyWindow::StopMessageLoop()
44: { 45: //Two stop marks
46:
47: m_bStop = true;
48: SendMessage(m_hWnd, WM_APPSTOP, 0, 0);
49: }
50:
51:
52: void NativeMessageOnlyWindow::StartMessageLoop()
53: { 54: m_bStop = false;
55: //Create the Windows
56: Create();
57:
58: MSG msg;
59:
60: //The traditional message loop
61: while (GetMessage(&msg, NULL, 0, 0))
62: { 63: TranslateMessage(&msg);
64: DispatchMessage(&msg);
65: }
66: }
67:
68: NativeMessageOnlyWindow::~NativeMessageOnlyWindow()
69: { 70: if (m_hWnd == nullptr)
71: return;
72: StopMessageLoop();
73: }
74:
75: LRESULT CALLBACK NativeMessageOnlyWindow::WndProc(HWND hWnd, UINT message, WPARAM wParam, LPARAM lParam)
76: { 77: //Find the correlated NativeMessageOnlyWindow instance
78: auto iter = s_windowsMap.find(hWnd);
79: if (iter == s_windowsMap.end()) //instance not found
80: { 81: OutputDebugString(L"NativeMessageOnlyWindow, a message can't be dispatched, HWDN not found.");
82: return DefWindowProc(hWnd, message, wParam, lParam);
83: }
84:
85: auto This = iter->second;
86:
87: if (message == WM_APPSTOP && This->m_bStop) //Is Stop
88: { 89: PostQuitMessage(0); //Send WM_QUIT to stop the message loop
90: s_windowsMap.erase(iter); //Remove the NativeMessageOnlyWindow instance from the hash map
91: This->m_hWnd = nullptr;
92:
93: return DefWindowProc(hWnd, message, wParam, lParam);
94: }
95:
96: //Trigger the callback call to the managed wrapper instance
97: return reinterpret_cast<LRESULT>(
98: This->_messageCallback->OnMessageReceivedCallback(IntPtr(hWnd), message, wParam, lParam).ToPointer());
99: }
100: }
101:
1:
2: using System;
3: using System.Runtime.InteropServices;
4: using System.Collections.Generic;
5: using System.Linq;
6: using System.Threading;
7: using System.Threading.Tasks;
8: using MessagePump;
9: using Microsoft.VisualStudio.TestTools.UnitTesting;
10:
11: namespace TestMessagePump
12: { 13: /// <summary>
14: /// Summary description for UnitTest1
15: /// </summary>
16: [TestClass]
17: public class UnitTestMessagePump
18: { 19: /// <summary>
20: ///Gets or sets the test context which provides
21: ///information about and functionality for the current test run.
22: ///</summary>
23: public TestContext TestContext { get; set; } 24:
25: [DllImport("user32.dll", CharSet = CharSet.Auto)] 26: static extern IntPtr SendMessage(IntPtr hWnd, UInt32 msg, UIntPtr wParam, IntPtr lParam);
27:
28:
29: [TestMethod]
30: public void TestMethodSendingMessages()
31: { 32: var messagePump = new MessageOnlyWindow();
33: var messageList = new List<WindowsMessage>();
34: var loopHasStarted = new ManualResetEventSlim(false);
35: // ReSharper disable InconsistentNaming
36: const uint WM_APP = 0x8000;
37: // ReSharper restore InconsistentNaming
38:
39: messagePump.MessageReceived += message =>
40: { 41: lock (messageList)
42: { 43: messageList.Add(message);
44: }
45: loopHasStarted.Set();
46: return IntPtr.Zero;
47: };
48: bool loopHasStopped = false;
49:
50: Task.Factory.StartNew(() =>
51: { 52: messagePump.StartMessageLoop();
53: loopHasStopped = true;
54: });
55:
56: Thread.Sleep(1000);
57: SendMessage(messagePump.WindowHandle, WM_APP, UIntPtr.Zero, IntPtr.Zero);
58: var result = loopHasStarted.Wait(1000);
59: Assert.AreNotEqual(result, false, "Timeout, a message hasn't arrived");
60:
61: SendMessage(messagePump.WindowHandle, WM_APP + 1, UIntPtr.Zero, IntPtr.Zero);
62: SendMessage(messagePump.WindowHandle, WM_APP + 2, UIntPtr.Zero, IntPtr.Zero);
63:
64: Assert.AreNotEqual(loopHasStopped, true,
65: "The message Loop should not be stopped until we call the StopMessageLoop");
66:
67: messagePump.StopMessageLoop();
68: Thread.Sleep(3000);
69: Assert.AreEqual(loopHasStopped, true,
70: "The message Loop should stopped since we have called the StopMessageLoop");
71:
72: var numberOfAppMessages =
73: messageList.Count(
74: message =>
75: message.Message == WM_APP || message.Message == WM_APP + 1 || message.Message == WM_APP + 2);
76:
77: Assert.AreEqual(numberOfAppMessages, 3, "There should be three WM_APP + x Messages in the list");
78:
79: }
80: }
81: }
82:
Download the VS 2010 solution from here.
Enjoy!
You didn't have to go to //BUILD/ to get the information and knowledge of the new Windows Platform APIs; you could watch all the sessions from the comfort of your home, and I encourage you to do so, however attending the conference helps obtaining a perception about the future of Microsoft and especially the future of Windows technologies and the reaction of the developers that attended the conference.
I didn't want to post about the conference before I have a clear picture, and believe me the picture was obscured. According to the first day keynote and the discussions on tweeter and the various mailing lists, I got it all wrong; however by digging into the implementation of WinRT, by talking with people from the Platform team and by attending the last day sessions, I got a better and clearer picture.
So what is Metro Style Application, what is WinRT, and should I learn C++ or HTML & JavaScript?
Let's start with the underlying system. Windows Run-Time (WinRT) is a new platform for using and consuming the Windows OS resources. It is a modern native layer on top of Win32 and the Windows Kernel that replaces Win32 APIs and other frameworks like MFC for the native world and many of the .NET classes and technologies for the managed environment.
- WinRT has a new type system, quite similar to the .NET Common Type System (CTS), however it is optimized for native code usage and for cross language calls with .NET, JavaScript and C++ in mind. The type system also provides rich type information that is based on the same metadata schema of .NET, however for native code there is no IL, GC and JIT.
- WinRT is heavily based on COM, however it is not IDL based, and we get all the goodies of a modern Object Oriented based technology such as classes, static functions, methods, properties, delegates, events and inheritance.
- WinRT APIs are either asynchronous for potentially long operations, or synchronous for fast non-blocking functions.
- WinRT provides the installation facilities as well as the environment activation and hosting.
- WinRT is based on components. This enables sharing implementation between languages.
- WinRT provides rich mechanism for inter Metro application communication through Contracts - as a developer you have the Launch Contract, Search Contract, Share Contract and many more. You can provide contract implementation or consume a contract.
Metro is the new client development platform. It is based on WinRT classes. Metro applications are fast, smooth, and safe. Metro application can use only "safe" APIs, which are the WinRT classes' members and a white list of other APIs. For .NET a special client profile is provided that contains only non (potentially) blocking calls and does not contain duplications for APIs provided by WinRT. To use sensitive user information such as location devices or a Web Cam, you need to declare these needed capabilities. Windows will ask the user for permission to run your code and use these capabilities (much like the standard for mobile applications). Metro Apps occupy the whole screen (chrome-less) and have a new life time management. They have to start quickly, they get suspended once they are in the background and background apps get terminated if the system resources are low. Each WinRT application runs as an STA COM component, and if there is a need to answer a new contract request, a new STA instance is lunched.
Now to some of the BIG concerns, what are the consequences of these new Windows mechanisms?
If you are a Windows C++ developer, it means that you get much better tooling and a framework that resembles the .NET development API. You need to learn the XAML concepts and the Metro UI object model. You also need to learn the C++ extensions for WinRT – although if you are familiar with C++/CLI, the extensions are very much the same. If you haven't upgraded your knowledge to C++ 11, you also need to handle this learning curve. The result is much better productivity, safe code, and fast & fluid user interface.
C++ is back, as Herb Sutter said in his talk – for fifteen years we didn't care about performance, but now that we pay for running our code in the cloud, or we target lightweight hardware such as slate devices and we want to preserve power, performance is a big issue again. C++ and WinRT are performance tuned and WinRT ref-count based resource management provides deterministic finalization which means that we don't hold unneeded resources.
If you are a C#/VB developer and productivity is more important to you than execution speed and low resource consumption, continue to do what you already do. Use the Metro profile with the .NET framework. WinRT provides all the metadata information that is needed to create and call WinRT objects, while .NET provides the GC, JIT, and all other mechanisms that make .NET so easy to use. To call a WinRT component .NET uses the same CCW/CRW mechanism that is used for COM interop, however the borrowed metadata mechanisms plus the new WinRT type system make this interop story much easier and faster. I hear a lot of complaints from .NET developers that WinRT "kills" many .NET technologies, however the exact opposite is the true! WinRT borrowed many of its concepts from .NET, its metadata, properties, delegates & events and others. Metro UI is based on XAML and the Silverlight/WPF object model. It is easier for a .NET developer to develop new metro application compared to a native C++ MFC developer.
If you are a web developer, JavaScript is a first class citizen for developing Metro style application on top of WinRT. You get the WinRT object model, and method names even appear in Camel Casing even though for C#/VB/C++ they appear as Pascal Casing.
If you are a UI/UX developer, Blend 5 and Visual Studio UI editor are upgraded to support Metro style application. You can also design HTML/CSS using Blend. You get to preserve all your old knowledge and practice while designing with the new Metro style. You also get many predefined UI templates.
If you are a Windows developer, many of the technologies that you know are there. Everything behind the scenes is COM, with major abstraction on top of COM. You can choose the language that you want or the one that you think is better for the job. You can develop C++ based components using native libraries such as boost and consume them from JavaScript or C#.
There are many new details to dive into now, but we also have the time. We are not even in the beta phase of the system and things will certainly change, but we are starting to see the tip of the iceberg.
I recently played with XNA. There is a game that I have developed many times, on the Commodore 64 as a kid, on the Commodore Amiga, later on Windows 9X with MFC, after that in .NET and Windows Forms, and now with XNA. The development experience in XNA was the best, and I can run my game on the phone, on Windows and on my XBOX.
The goal of the game is to have higher score 
. Each player moves only in a specific direction (vertical or horizontal). In each turn a player move the token (S shape) and take a number (replace a number with the token), the value of the number is added to the player score.
One interesting thing about the game is that I usually lose when I play with it. The game thinking is base on Min-Max with Alpha-Beta pruning algorithm. The heuristics is very simple: player1 score minus player2 score. The computer/XBox/Phone is thinking while the player is thinking.
When I win, I think what I did wrong, When I lose, I proud that the algorithm works, but I then I think how come a so simple algorithm can be so smart…
The only way to win the computer is if you get a board that has a better number setting in your playing direction, how do I know that? because I added an option to play the same game with the opposite direction…
The game is working and is almost done, but I have more ideas that I want to implement before I will upload it to the Windows 7 Application Store, for example a two players game using push notifications.
If you familiar with C# and .NET, you just need a good kick start for XNA and you there, and XNA is not just for games, it is the easiest platform to develop 2D and 3D based graphics and animations on top of Windows, Windows Phone and XBOX, and you can host an XNA Windows in your WPF/WinForm application.
More information about XNA development can be found on Josh’s blog, and if you really want to know how, register to Josh’s XNA Expert Day Session.
At the beginning of 2005 Herb Sutter had an article stating that the developer’s free lunch is over. We had an assumption that more transistors in the CPU imply better application execution speed. The CPU executes the code in a sequential manner hence the performance of the CPU-bound code is directly related to CPU frequency. This used to be our “Free Lunch”: an old program runs faster on a new CPU. Using this assumption with modern low power consumption multi-core CPUs is wrong, we might even find that an old program runs slower on a new CPU. Since performance is no longer tied to CPU frequency we need to leverage parallelism. To some extent this is not a new concept, multiprocessing systems exist for years. For example Windows NT was designed back in 1989 as a Symmetric Multiprocessing Operating System. Hyper threading CPUs exist at least half a decade. But do we get better performance by moving from one-way machine to 16-ways? We need a better abstraction!
Our software design can be based on multi-threading execution. We can align the number of threads to the number of cores, but does this mean that if we double the cores, we double the performance? Probably not! There are many obstacles that we need to overcome. It is not easy to split a task to many threads. Creating threads is a costly operation. Kernel based context switch implies more overhead. We need to make sure that the number of running thread concurrently will not be more than the number of cores, however if a thread gets blocked we need to free or create another thread to run to utilized the free core. Add to that the fact that systems with many cores have NUMA architecture, which means that allocating and affinity of memory to threads and core can gain more performance.
Parallelism is hard, instead of inventing the underlying abstraction; we want to use existing well-known ones. In this post we will see how the C++ Concurrency Runtime (ConcRT) and Parallel Pattern Library (PPL) provide the needed abstraction and bring us back the free lunch. Using these abstractions we will gain performance from running the same application on many cores. We will also compare the new ConcRT libraries to the .NET Task Parallel Library (TPL) and see that the concept are quiet similar, however C++ has a better runtime (Resource Manager and Task Scheduler) that taking NUMA into consideration and can result better performance on such architectures.
To show that throwing more cores result better performance we will calculate prime numbers in a loop. We want to find the highest prime number that we can in 10 seconds. We will start with one core and do the same algorithm throwing more cores. Since I am using a Core i7 Hyper-threaded enabled machine, we will see some degradation in scalability when we will reach a high number of cores (actually virtual CPUs since it is hyper-threaded).
Bear in mind that finding the highest prime number is non-linear problem. I.E, finding a 10 digits number will take much more time than finding a 5 digits prime number. So it will be hard to deduce the scalability factor. What we will see is that adding more cores brings back our free lunch!
To keep contention to the minimum we will use the combinable<T> class. This class provides a local copy for each thread. In our case each thread will hold its maximum finding and then using the combine() method, we will pick the maximum number from all the threads. To handle the time limit, we will use the task task_group.cancel() method to finish the current parallel loop execution. To limit the range of numbers, we will use groups of 8192 numbers in parallel at a time.
The C++ Code:
// ----------------------------------------------------------------------
// <copyright file="GettingBackTheFreeLunch.cpp" company="CodeValue">
// Copyright (c) 2011 by CodeValue Ltd. All rights reserved
// </copyright>
//
// http://codevalue.com
// Licensed under the Educational Community License version 1.0 (http://www.opensource.org/licenses/ecl1)
// This example was written as a demonstration of principles only,
// as part of the Windows Concurrent programming course for C++ developers
//
// ------------------------------------------------------------------------
#include "stdafx.h"
#include <Windows.h>
#include <ppl.h>
#include <concurrent_vector.h>
#include <iostream>
#include <functional>
#undef max
using namespace std;
using namespace Concurrency;
ULONGLONG FindLargestPrimeNumberInTime(DWORD concurrencyLevel, DWORD timeInterval);
int _tmain(int argc, _TCHAR* argv[])
{
SYSTEM_INFO systemInformation;
::GetSystemInfo(&systemInformation);
wcout << L"Number of logical processor: " << systemInformation.dwNumberOfProcessors << endl;
for (DWORD concurrencyLevel = 1; concurrencyLevel <= systemInformation.dwNumberOfProcessors; ++concurrencyLevel)
{
DWORD timeInterval = 1000 * 10;
ULONGLONG largetPrimeNumber = FindLargestPrimeNumberInTime(concurrencyLevel, timeInterval);
wcout << L"Result: " << largetPrimeNumber << L" in " << timeInterval/1000 << L" seconds using " << concurrencyLevel << L" Logical Processors" << endl;
}
return 0;
}
bool IsPrime(ULONGLONG number, DWORD tickCountLimit)
{
if ((number & 1) == 0)
return false;
ULONGLONG limit = static_cast<ULONGLONG>(sqrt(static_cast<double>(number)));
for (ULONGLONG n = 3; n <= limit; n += 2)
{
if ( (number % n == 0) || (::GetTickCount() > tickCountLimit))
return false;
}
return true;
}
ULONGLONG FindLargestPrimeNumberInTime(DWORD concurrencyLevel, DWORD timeInterval)
{
ULONGLONG result = 3;
SchedulerPolicy policy;
policy.SetConcurrencyLimits(concurrencyLevel, concurrencyLevel);
CurrentScheduler::Create(policy);
task_group taskGroup;
taskGroup.run_and_wait([&]
{
DWORD dueTime = ::GetTickCount() + timeInterval;
combinable<ULONGLONG> maxPrime([]()->ULONGLONG { return 3; });
ULONGLONG slice = 8192;
for (ULONGLONG range = 3; range < ~static_cast<ULONGLONG>(0); range += slice)
{
parallel_for<ULONGLONG>(range,range + slice, 1, [&](ULONGLONG n)
{
if (IsPrime(n, dueTime))
maxPrime.local() = max(maxPrime.local(), n);
if (::GetTickCount() > dueTime)
taskGroup.cancel();
});
result = maxPrime.combine(max<ULONGLONG>);
}
});
return result;
}
The result is:
Number of logical processor: 8
Result: 16252919 in 10 seconds using 1 Logical Processors
Result: 19365887 in 10 seconds using 2 Logical Processors
Result: 23945203 in 10 seconds using 3 Logical Processors
Result: 26845127 in 10 seconds using 4 Logical Processors
Result: 29343719 in 10 seconds using 5 Logical Processors
Result: 31547357 in 10 seconds using 6 Logical Processors
Result: 33079289 in 10 seconds using 7 Logical Processors
Result: 26673149 in 10 seconds using 8 Logical Processors
Press any key to continue . . .
To show the C# free lunch and to compare the results, I implement the same program using .NET TPL. Instead of combinable, the .NET parallel programing sample pack provides the ReductionVariable class. To stop the loop I use the loopState loop controlling mechanism. To control parallelism I use the ParallelOptions.MaxDegreeOfParallelism property:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace GettingBackTheFreeLunch
{
class Program
{
static void Main()
{
Console.WriteLine("Number of logical processor: {0}", Environment.ProcessorCount);
for (int concurrencyLevel = 1; concurrencyLevel <= Environment.ProcessorCount; ++concurrencyLevel)
{
const int timeInterval = 1000 * 10;
long largetPrimeNumber = FindLargestPrimeNumberInTime(concurrencyLevel, timeInterval);
Console.WriteLine("Result: {0} in {1} seconds using {2} logical processors",
largetPrimeNumber, timeInterval / 1000, concurrencyLevel);
}
}
static bool IsPrime(long number, long tickCountLimit)
{
if ((number & 1) == 0)
return false;
var limit = Math.Sqrt(number);
for (long n = 3; n <= limit; n += 2)
{
if ((number % n == 0) || (Environment.TickCount > tickCountLimit))
return false;
}
return true;
}
private static long FindLargestPrimeNumberInTime(int concurrencyLevel, long timeInterval)
{
var parallelOption = new ParallelOptions
{
MaxDegreeOfParallelism = concurrencyLevel
};
long dueTime = Environment.TickCount + timeInterval;
var maxPrime = new ReductionVariable<long>(() => 3);
const long slice = 8192;
for (long range = 3; range < long.MaxValue; range += slice)
{
Parallel.For(range, range + slice,
parallelOption, (n, loopState) =>
{
if (loopState.IsStopped)
return;
if (IsPrime(n, dueTime))
maxPrime.Value =
Math.Max(maxPrime.Value, n);
if (Environment.TickCount > dueTime)
loopState.Stop();
});
if (Environment.TickCount > dueTime)
break;
}
long result = maxPrime.Reduce(Math.Max);
return result;
}
}
}
The result of the C# code is:
Number of logical processor: 8
Result: 7946903 in 10 seconds using 1 logical processors
Result: 12932119 in 10 seconds using 2 logical processors
Result: 16153307 in 10 seconds using 3 logical processors
Result: 19601821 in 10 seconds using 4 logical processors
Result: 21543757 in 10 seconds using 5 logical processors
Result: 23133361 in 10 seconds using 6 logical processors
Result: 24362383 in 10 seconds using 7 logical processors
Result: 25337269 in 10 seconds using 8 logical processors
Press any key to continue . . .
The C++ code is a little bit faster, it is hard to tell how much faster due to the non-linearity of the problem. We also need to check it on a real multi-core NUMA based server to have a better comparison between the C++ and the C# code. The nice thing about the C# and the C++ implementation is that we have frameworks that can bring us back the free lunch.
Visual Studio 2010 brings back the power of C++. The new language capabilities and optimized STL libraries, the developers productivity as well as the resulting executable performance are much higher. In this post I’d like to present one of the abilities of the ConcRT Asynchronous Agents Library. Agent is based on the Actor Model and is one of the building block of a Concurrent Dataflow Network. The idea behind the Asynchronous Agent Library is that you build a network from agents and message blocks. Messages are sent from agent and blocks to other agents and . The ConcRT scheduler is responsible for dispatching virtual processors (threads) to agents and message blocks:
• An Agent is an asynchronous object that communicates through message passing
• Message Blocks participate in message-passing which transports from source to target
• Message encapsulates the state that is transferred between message blocks
This screen shot is taken from a new C++/WIn32 Concurrent Course of CodeValue
Parallelism is achieved by dividing processing tasks into discrete agents. Each agent has a single, well-defined task. Each agent operates on a dedicated memory buffer. Underline the runtime uses lightweight tasks to dispatch messages to agents. You don’t need to synchronize anything. You just need to identify your agents, and the memory buffers that build the network. In this post I want to focus on the Memory Blocks (Buffers). Usually memory blocks are like queues. They receive and contain messages. However unlike an ordinary queue, they can have special characters and they serves as message dispatchers.
The library defines tow important interfaces (abstract struct in C++): the ISource & the ITarget. Memory Blocks implement these interfaces to enable messages propagation. There are many predefine Message Blocks, and you can even define your own. The Technical Computing team that is responsible for ConcRT among many other interesting things has released a Sample Pack that you can download and use. The sample pack is like CTP and contains extra features including more predefined Message Blocks.
The following table provides some information about the various Message Blocks (include the ones that are in the Sample Pack sp):
The simplest Message Block is the unbounded_buffer, It can be connected to many sources, holds many messages and can dispatch those messages to many targets. The overwite_buffer can do the same but stores only the last message it has received. Timer is a source that sends message when a period of time is elapsed. To send a message to a buffer we use the global send function. To receive a message from a buffer we you, yes you are correct, the receive function. We can use asend to asynchronously send a message and try_receive if we don’t want a blocking wait for a message.
With all this knowledge, lets create a Message Block Network that solves a logic circuit of Full-Adder:
We are going to use the join_transform Message Block that is part of the Sample Pack. the join_transform is a combination of the join & transformer message blocks. It receives messages from many sources and combine them to one output by applying your transform function (can be Lambda function). We will have three functions for And, Xor and Or. We will use the overwite_buffer for our inputs (A, B, Cin) and we use unbounded_buffer as a connector for outputs that have more then one target.
To have better abstraction we are going to use typedefs .
this is the code:
// ----------------------------------------------------------------------
// <copyright file="DataflowLogicCircuit.cpp" company="CodeValue">
// Copyright (c) 2011 by CodeValue Ltd. All rights reserved
// </copyright>
//
// http://codevalue.com
// Licensed under the Educational Community License version 1.0 (http://www.opensource.org/licenses/ecl1)
// This example was written as a demonstration of principles only
//
// ------------------------------------------------------------------------
// DataflowLogicCircuit.cpp
// Implementing Full-Adder
// By Alon Fliess
#include "stdafx.h"
#include <agents.h>
#include <iostream>
#include "agents_extras.h"
using namespace std;
using namespace Concurrency;
using namespace samples;
int _tmain(int argc, _TCHAR* argv[])
{
auto and = [](vector<bool> input) -> bool {return input[0] && input[1]; };
auto or = [](vector<bool> input) -> bool {return input[0] || input[1]; };
auto xor = [](vector<bool> input) -> bool {return input[0] ^ input[1]; };
typedef overwrite_buffer<bool> Switch;
typedef join_transform<bool, bool,greedy> gate;
Switch A,B,Cin;
gate x1(2, xor);
overwrite_buffer<bool> x1Output;
x1.link_target(&x1Output);
gate x2(2, xor);
gate a1(2, and);
gate a2(2, and);
gate o1(2, or);
A.link_target(&x1);
A.link_target(&a2);
B.link_target(&x1);
B.link_target(&a2);
Cin.link_target(&x2);
Cin.link_target(&a1);
x1Output.link_target(&x2);
x1Output.link_target(&a1);
a1.link_target(&o1);
a2.link_target(&o1);
for (int a = 0; a <= 1; ++a)
{
for (int b = 0; b <= 1; ++b)
{
for (int cin = 0; cin <=1; ++cin)
{
send(&A, a != 0);
send(&B, b != 0);
send(&Cin, cin != 0);
wcout << a << L" + " << b << L" + " << cin <<
L" = " << receive(&x2) << L" (" << receive(&o1) << L")" << endl;
}
}
}
return 0;
}
The result:
If you are a C++ developer and find this post useful please comment about that.