Shared Memory Queue, Adaptive pthread_mutex, and Dynamic Tracing

Friday, January 22, 2016

This blog post is also on GitHub in its entirety. If you prefer to read it there along with the code, I won't mind. Go ahead. In one of my recent training classes, I was asked to demonstrate some practical uses of shared memory. My knee-jerk reply was that shared memory can be used for inter-process communication and message-passing. In fact, most IPC mechanisms are based on shared memory in their implementation. The question was whether it's worth the effort to build a message-passing interface on top of shared memory queues, or whether sockets or pipes could produce a better result...
2 comments

Myth-Busting: Memory-Mapped Files and Shared Memory on Windows

Thursday, January 21, 2016

I am often asked why memory-mapped files can be more efficient than plain read/write I/O calls, and whether shared memory is slower than private memory. These seemingly unrelated mechanisms share a common implementation in the Windows kernel, known as section objects or file mapping objects. Yes, this shared implementation powers memory pages that are shared across multiple processes (by name) as well as file regions mapped to memory pages (even in a single process). If you're interested in a thorough discussion of how section objects work, I must refer you to Windows Internals, 6th Edition. But if you're only here for...
no comments

Windows Process Memory Usage Demystified

Tuesday, January 5, 2016

"How much memory is your process using?" -- I bet you were asked that question, or asked it yourself, more times than you can remember. But what do you really mean by memory? I never thought it would be hard to find a definitive resource for what the various memory usage counters mean for a Windows process. But try it: Google "Windows Task Manager memory columns" and you'll see confusing, conflicting, inconsistent, unclear explanations of what the different metrics represent. If we can't even agree on what "working set" or "commit size" means, how can we ever monitor our Windows...
10 comments

Wrapping Up DotNext 2015

Saturday, January 2, 2016

A few weeks ago, I had the honor of being invited to speak at DotNext 2015, Russia's only .NET conference and one of the leading developer conferences in the country. As some of my readers probably know already, I was born in the USSR, so I speak Russian with a heavy Israeli accent but can understand both written and spoken Russian very well. The fact it was my wife's birthday and we could elope for a weekend of wintery weather and hardcore CLR internals only added to my resolve. I proposed two talks, and the organizers had such difficulty picking...
no comments

Live360! and BuildStuff Talks: SIMD, Visual Studio Diagnostic Hub, and Swift

Wednesday, December 9, 2015

I'm writing this on the plane back home from a week-long trip to Orlando, Vilnius, and Kiev, where I had the chance to speak at Live360! and BuildStuff; I've just counted and it's my tenth flight in three weeks, which is quite insane. But this is my second-to-last trip for 2015 -- the last one is going to be in December to DotNext Moscow. SIMD In this talk, we discussed vector registers and instructions that you could use from other languages like FORTRAN and C++ for more than 15 years. Starting from the MMX instruction set extensions in the 1997...
no comments

Wrapping Up Software Architect 2015

Tuesday, October 27, 2015

I travel to a lot of conferences, but among the ones I like the most are Software Architect and DevWeek. I'm writing this post on the flight back home from Software Architect, where I had the pleasure of delivering a workshop and three talks. If you attended the conference, thanks a lot for coming and I hope you find the materials useful; if you haven't been to the conference, I expect to see you next year! Haskell  My first talk was an introduction to Haskell for developers with no prior experience in functional programming. For me personally, Haskell is not...
no comments

Large Win32 Heap Allocations Go Directly to VirtualAlloc

Friday, October 23, 2015

The Windows heap manager was designed to avoid the overhead of having to allocate virtual memory directly with VirtualAlloc, among other things. If you only need a 20-byte object, it's a waste to call a system service (involving a user-kernel transition) and allocate a full page. The heap manager avoids that overhead by managing large blocks of virtual memory in user mode---it is implemented in ntdll.dll. However, when you allocate particularly large blocks of memory (>= 512KB at the time of writing), the heap manager doesn't see a reason to interfere, so it just forwards your request to VirtualAlloc. It still knows about...
2 comments

Wrapping Up TechDays Sweden 2015

Thursday, October 22, 2015

This is my second TechDays this year--I've had the pleasure of visiting the Netherlands and now also Sweden to speak at TechDays on a variety of topics. Both events had quite a special atmosphere with a lot of activity, a lot more session slots than usual at software conferences, and a great selection of international speakers (except for your humble servant, of course). Even though most talks were in languages I don't understand, I really enjoyed my time. But I'm guessing you're here because you want to put your hands on materials from my two talks--so here they are, without...
no comments

More on MiniDumper: Getting the Right Memory Pages for .NET Analysis

Wednesday, September 30, 2015

In my previous post on MiniDumper, I promised to explain in more detail how it figures out which memory ranges are required for .NET heap analysis. This is an interesting story, actually, because I tried a couple of approaches that failed before coming up with the final idea. Basically, I knew that a dump with full memory contains way more information than is necessary for .NET dump analysis. Even if you need the entire .NET heap available, you typically don't need a bunch of other memory ranges: executable code, Win32 heaps, unused regions of thread stacks, and so much more. My...
one comment

Materials from TechDays Netherlands 2015

Monday, August 31, 2015

Oops! This was sitting in my queue for several months now, and I just noticed it needs to be published. But better late than never I guess. Here goes: I've been lucky enough to be invited to speak at TechDays Netherlands again this year. This time I was asked to do four talks on some of my favorite subjects -- performance optimization, debugging, and diagnostics. Same as last year, the conference was impeccably organized. I'm really looking forward to next year's TechDays :-) In the meantime, here are the materials from my talks. Making .NET Applications Faster My usual favorite on improving...
7 comments