Diagnosing Native Memory Leaks with ETW and WPA

Tuesday, December 2, 2014

As a followup to my previous post on native memory leaks, here's a quick walkthrough for diagnosing memory leaks using Event Tracing for Windows. The process is fairly simple. The Windows heap manager is instrumented with ETW traces for each memory allocation and deallocation. If you capture those over a period of time (when your application is leaking memory), you can get a nice report of which blocks were allocated during the trace period and haven't been freed. If you also ask ETW to capture the call stack for allocation events, you can see where the application is allocating...
one comment

Native Memory Leak Diagnostics with Visual Studio 2015

Monday, December 1, 2014

The Current Landscape of Native Memory Diagnostics Leak diagnostics is a nasty business in native applications. There have been many attempts at solving this problem automatically. To name a few: The CRT Debug Heap (which is no longer used by default in Visual Studio 2015! - See update below.) can help identify memory leaks by associating each allocation with additional data on the allocating source file and line number. At program exit (or whenever a special CRT function is called), all blocks that haven't been freed are printed out. This has been around forever. The problem is that you need to...
8 comments

Garbage Collection and .NET Debugging at Build Stuff

Tuesday, November 25, 2014

I spent most of last week at Build Stuff, a really cool software conference in Vilnius, Lithuania. The conference was great with a really exciting atmosphere: energized, passionate developers having conversations and playing table tennis in the hallways during the day, and drinking lots of beer in the evenings. Even the weather was quite nice -- there was only a little snow, and temperatures didn't drop below -1 Celsius, which means we could walk around the old town's historical landmarks; grab some sushi, ribs, and beer; and do some window shopping. So, a great success! I was invited to deliver...
no comments

A Loop of Nested Exceptions

Monday, November 17, 2014

It was a pretty incredible coincidence. Only a few days apart, I had to tackle two problems that had to do with nested exception handlers. Specifically, an infinite loop of nested exceptions that led to a stack overflow. And that's a pretty fatal combination. A stack overflow is an extremely nasty error to debug; a nested exception means the exception handler encountered an exception, which can't be pretty; and to add insult to injury, a stack corruption was also involved behind the scenes. Read on to learn some more about the trickiness of diagnosing nested exceptions and what can...
4 comments

TechEd Europe 2014: Mastering IntelliTrace in Development and Production

Monday, October 27, 2014

I'm flying to TechEd Europe tomorrow, and decided to run an experiment and post my slides and demos before the session. Why the weird timing? Well, after giving the schedule a cursory glance, there are so many great sessions! It's really hard to pick a session based on the short conference abstracts, and I wouldn't want anyone to come to my session if they aren't absolutely sure it's a topic they care about. My talk is titled Mastering IntelliTrace in Development and Production. I love IntelliTrace and use it a lot, but it still remains a fairly obscure Visual Studio...
2 comments

DevConnections 2014: IntelliTrace, Diagnostics Hub, and .NET Production Debugging

Saturday, September 20, 2014

I'm flying back home from DevConnections 2014, which was great! Vegas was hot and dry as usual, but I actually managed to carve out some time in my schedule to see KA, which was really nice. (Plus, the conference was at the Aria resort, which is located smack in the middle of the strip, and is overall much nicer than Mandalay Bay where we were last year. I really liked the hotel room automation control. For example, I had an alarm clock set up to open the curtains, turn on the TV to a quiet music channel, and even...
no comments

A Motivating Example of WinDbg Scripting for .NET Developers

Tuesday, August 5, 2014

WinDbg scripting gets a pretty bad name -- its somewhat contrived syntax, weird limitations, and hard to decipher expressions being the common culprits. In some cases, however, WinDbg scripts can be a very effective and reliable tool for extracting information from memory and processing it in a meaningful way. This post offers a simple example that hopefully will be useful as you begin to explore WinDbg scripts. For a more thorough explanation and more complex scripts, make sure to check my past posts on traversing std::vector and std::map. Let's set the stage with a simple console application that creates a number...
3 comments

Tracking Unusable Virtual Memory in VMMap

Tuesday, July 22, 2014

VMMap is a great Sysinternals tool that can visualize the virtual memory of a specific process and help understand what memory is being used for. It has specific reports for thread stacks, images, Win32 heaps, and GC heaps. Occasionally, VMMap will report unusable virtual memory, which is not the same as free memory. Here's an example of a VMMap report for a 32-bit process (which has a total of 2GB virtual memory): Where is this "unusable" memory coming from, and why can't it be used? The Windows virtual memory manager has a 64KB allocation granularity. When you allocate memory directly...
one comment

Identifying Specific Reference Type Arrays with SOS

Thursday, May 1, 2014

When you're looking for arrays of a specific type using SOS, you might notice a weird phenomenon. Value type arrays (such as System.Int32) will be shown properly regardless of which command you use, but reference type arrays (such as System.String) exhibit some weird behavior. Here's an example: 0:000> !dumpheap -stat Statistics: MT Count TotalSize Class Name ... 00007ffecf435740 2 304 System.Byte 00007ffecf4301c8 2 320 System.Threading.ThreadAbortException 00007ffecf4327d8 11 ...
no comments

Diagnosing a Non-Paged Pool Leak with Asynchronous I/O

Thursday, February 20, 2014

I spent a few hours last week chasing a non-paged pool leak caused by a simple Win32 application. After some divide-and-conquer work, we were able to pinpoint the line of code causing the leak -- a seemingly innocent WSARecv call that performs an asynchronous socket receive. How can a user-mode application cause a non-paged pool leak that quickly accumulates to dozens of megabytes of kernel memory? Read on for the details. If you'd like to replicate this problem yourself and experiment with the diagnostic process described below, use the following gist. It's 54 lines of code including error handling and #includes. Capturing...
no comments