A Neat Stack Corruption, or Reverse P/Invoke Structure Packing with Output Parameters

Tuesday, August 18, 2015

I know, I'm working hard on beating my record for longest post title ever. I also thought of adding a random Win32 API to the title, say CoMarshalInterThreadInterfaceInStream or AccessCheckByTypeResultListAndAuditAlarmByHandle. But I didn't, so here we are. What was I saying? Oh yeah, a neat stack corruption I spent a couple of hours chasing last week. I was doing my usual reverse P/Invoke where I call a Windows API and pass a delegate as a callback. There's a bunch of APIs in Win32 that take callbacks, but for the sake of this post let's take a look at a very simple example...

Hosting a .NET DLL as an Out-Of-Process COM Server (EXE)

Tuesday, January 7, 2014

I recently came across the need of hosting a .NET assembly (DLL) as an out-of-process COM server. While .NET ships with excellent support for COM interoperability, this particular scenario isn't directly addressed by the interop layer. While there are some fairly complex solutions such as COM+ hosting or custom class factory registration, I would like to show you how to accomplish this task using dllhost.exe, the default COM surrogate process, with just a handful of registry modifications. Why? The reason I was looking into doing this is quite simple. Suppose you have a 32-bit only unmanaged DLL that you can't recompile...
tags: , ,

Back To Basics: COM Interop and No-PIA

Saturday, December 28, 2013

Almost every week, I meet developers who hate COM. Some hate it with the heat of a million suns. Some have never used it, but heard horror stories from grey-haired colleagues. Other survivors still remember their experience fighting ATL linker errors in Visual C++ 6.0. And you know what, you can hate COM all you want. But it's a little like hating electricity. Or the Internet. Because -- on Windows -- COM powers almost everything. Office. The Windows Shell. This beautiful cozy Windows Runtime APIs that you're enjoying from JavaScript and C#. And did you know that the .NET...
tags: , ,

Workshops at Sela Developer Practice, December 2013: Improving .NET Performance and .NET/C++ Interop Crash Course

Thursday, December 19, 2013

In addition to my three breakout sessions, I've also had the pleasure of delivering two workshops at the Sela Developer Practice: Improving .NET Performance and .NET/C++ Interop Crash Course. Although these workshops are quite time-tested, I always try to add new materials and tools to make them more interesting for both myself and the audience. There's also constant interest in these topics -- I had 110 people registered for the performance workshop and more than 40 people at the interop course. In the performance workshop, we cover various performance measurement tools. I always try to squeeze in new tools in...
no comments

P/Invoke with C++ bool Return Values

Thursday, January 17, 2013

I encountered an interesting gotcha today, which I thought would be interesting to share with you. The symptoms were the following: I wrote a simple C++ function exported from a DLL, and tried to invoke it using a P/Invoke wrapper. The C++ function returns a bool, and was declared as follows (the code was simplified for expository purposes): extern "C" __declspec(dllexport) bool IsPrime(int n) {     if (n <= 1) return false;     if (n == 2) return true;     if (n % 2...
tags: ,

Diagnosing Memory Leaks in Managed Windows Store Apps

Monday, October 15, 2012

There is so much material on the web (and even on this blog) about memory leak diagnostics in managed code, and a considerable number of tools that make diagnostics increasingly easier. Modern memory profilers can open application dumps, attach to live processes, display live memory graphs, compare snapshots, identify problematic retention patterns, and so much more. Unfortunately, these tools presently don’t work with Windows Store apps. Moreover, the UI model of Windows Store apps poses a significant challenge in diagnosing many UI-related memory leaks, such as composite UI controls retaining objects embedded in them, or UI elements retaining...

Managed-Unmanaged Interoperability in the 2010s: Part 3, C++/CLI and Other Options

Monday, January 30, 2012

What if P/Invoke is not enough? Hold on, why should it not be enough? What needs could we possibly have but calling global C-style functions exported from DLLs? Well, suppose you want to use a C++ class exported from a DLL. Or maybe the C++ class is not yet exported and you are looking for a way to make it available to managed code. The problem with a C++ class as opposed to a global function is that … it is not a global function. For example, a typical C++ class with a constructor and a couple...
no comments

Managed-Unmanaged Interoperability in the 2010s: Part 2, P/Invoke – Unmanaged to Managed

Saturday, January 28, 2012

We’ve got P/Invoke doing some heavy lifting of managed-to-unmanaged signature translation. The opposite direction is surprisingly easy as well. Typically, you would encounter the need for unmanaged code to call managed code as part of a callback. (There is the scenario where unmanaged code is the interop initiator, which we will discuss in another post.) In the unmanaged world, callbacks are function pointers; in the managed world, callbacks are delegates. Recall that a delegate “knows” not only the method that needs to be called, but also the target object – and indeed, you can create delegates that...
one comment

Managed-Unmanaged Interoperability in the 2010s: Part 1, P/Invoke – Managed to Unmanaged

Tuesday, January 24, 2012

With Windows 8 around the corner, managed code slowly taking over legacy systems written in C++, games developed in a mixture of .NET and C++, and the rest of this technology soup – I thought it would be a good time to provide a quick refresher of the available interoperability mechanisms between managed and unmanaged code. Nothing here is very new, but I get so many questions about it that at the very least I would have something to refer people to. P/Invoke is best suited for managed code invoking global functions exported from C-style DLLs....
no comments

P/Invoke Stack Imbalance MDA

Wednesday, January 18, 2012

More than a year after writing my first post touching on the subject of Managed Debugging Assistants (MDA) through the “Callback on Garbage Collected Delegate” case study, it’s time for a brief mention of another useful MDA – “P/Invoke Stack Imbalance”. This MDA fires whenever a P/Invoke call causes an imbalance on the stack. What does a stack imbalance mean? The top of the thread’s stack is pointed to by the ESP register (RSP on x64), and a stack imbalance occurs if the ESP value before making a function call is not the same as the ESP value...
no comments