Diagnosing a Non-Paged Pool Leak with Asynchronous I/O

Thursday, February 20, 2014

I spent a few hours last week chasing a non-paged pool leak caused by a simple Win32 application. After some divide-and-conquer work, we were able to pinpoint the line of code causing the leak -- a seemingly innocent WSARecv call that performs an asynchronous socket receive. How can a user-mode application cause a non-paged pool leak that quickly accumulates to dozens of megabytes of kernel memory? Read on for the details. If you'd like to replicate this problem yourself and experiment with the diagnostic process described below, use the following gist. It's 54 lines of code including error handling and #includes. Capturing...
no comments

Back to Basics: Mastering Visual Studio Breakpoints

Tuesday, November 12, 2013

I know what you’re thinking right now. Breakpoints? Seriously? What’s there to master about breakpoints? You hit F9 and then you stop at the line of code. If that’s what you’re thinking, this post is for you: read on :-) Visual Studio offers a fairly rich set of breakpoint types and actions that can make your debugging experience more comfortable and productive. We will look at four kinds of breakpoints in this post. Conditional Breakpoints When defining a breakpoint, you can associate it with a condition (right-click the breakpoint and choose...
tags: ,
no comments

Talks from DevConnections 2013: Advanced Debugging with WinDbg and SOS, Task and Data Parallelism, and Garbage Collection Performance Tips

Thursday, October 10, 2013

I'm falling behind in documenting all my travels this fall :-) In the beginning of the month I flew out to Vegas for IT/DevConnections, which was my second Las Vegas conference this year. I've been there for just 48 hours, but it was enough time to deliver three talks, meet fellow speakers, and even have a few meaningful chats with attendees about the future of .NET and production debugging techniques. You can find my presentations below -- the last couple of slides of each presentations have some additional references and books that might be useful if you want to expand...

Announcing Tracer: A Generic Way to Track Resource Usage and Leaks

Tuesday, September 10, 2013

Tracer is a WinDbg extension I wrote last month to diagnose a resource leak that is not covered by well-known facilities like !htrace or UMDH. Tracking any resource leak starts with understanding where you are acquiring the resource and neglecting to release it – and with Tracer, you can do this for any kind of resource. Download Tracer and review its source code. The basic process of hunting for resource leaks is quite simple. For example, consider what UMDH does on your behalf. UMDH enables support in the operating system (specifically, in the Heap Manager...
no comments

Obscure WinDbg Commands, Part 4

Monday, August 26, 2013

In this final installment, we will review some miscellaneous commands that can make your life a bit easier. First, the .wtitle command. This command changes the title of the WinDbg window. It’s simple, sure, but makes it that much easier to work when you have multiple WinDbg windows open. Another command that helps with command discovery is .cmdtree. This command takes a specially-formatted text file and displays a tree-like menu that will execute these commands for you (yes, somewhat akin to the .dml_start command which we have seen previously). For example, given my command...
one comment

Obscure WinDbg Commands, Part 3

Monday, August 19, 2013

In today’s installment, we’ll take a look at two commands that make it easier to trace through program execution. The first command is wt, which traces through all the function calls performed in a certain code path and formats nice statistics illustrating what happened during that function’s execution. wt has a bunch of options that I won’t be showing here, but the general idea is that you let it trace through a lot of unfamiliar code and display statistics on what was going on in that code. For example, here I used a switch to filter out...
one comment

Obscure WinDbg Commands, Part 2

Thursday, August 15, 2013

In today’s post, we’ll take a look at some of the options available to us when using DML (Debugger Markup Language). DML is a very simple markup language that helps discover and execute new commands based on the output of existing commands. Many WinDbg commands (and extension commands as well) have support for DML. For example, here’s the lm D command, which displays DML output: In the command output above, when I clicked the “SillyThreadPool” link, the debugger executed another command for me, lmDvmSillyThreadPool, which displays module information. Again, there’s a bug of links that help...
one comment

Obscure WinDbg Commands, Part 1

Monday, August 12, 2013

I’m starting a short post series today covering some obscure WinDbg commands. Some of these are fairly useful, others are mostly good for extracting “wows” from your coworkers. Still, there’s a lot of ground to cover. Today’s two commands deal with inspecting thread call stacks. Real-world processes might have hundreds of threads in them, and figuring out who’s who can be pretty time-consuming. This is especially true if you are using the thread pool (.NET, Win32, or even your own), which often means you have dozens of thread pool threads waiting idly for work. The !uniqstack command...
no comments

Setting Up Symbols for Offline Environments

Thursday, August 8, 2013

Yawn. That was my first reaction too, but turns out many developers around the world (including me in the not-so-distant past) work in a disconnected environment with no access to the Microsoft public symbol server or a symbol proxy. This post is for you. Microsoft offers a set of symbol packages for each major Windows version and service pack. If your system administrator does not install updates, ever, this is good enough for you. You can find the symbol packages here. If you are getting sporadic updates, chances are that a large number of Windows binaries have been touched by these...
no comments

Searching and Displaying C++ Heap Objects in WinDbg

Monday, August 5, 2013

This is something I am pretty excited to tell you about. But first, some motivation. In managed applications, there’s a huge number of tools and ways to inspect the managed heap contents. You can use a memory profiler to see references between objects and inspect individual objects. You can use WinDbg with the SOS extension to dump all objects of a particular type and execute additional scripts and commands for each object. You can even write C# code that uses the ClrMd library to parse heap contents and write your own diagnostic tools. C++ has nothing of...