January 2009 - Posts
To set context, a preview handler in Windows Vista and/or Outlook 2007 is a component that provides a preview window for a specific file type (or set of file types). For example, the default preview handler for .jpg files will display a preview of the image in the Windows Explorer or Outlook preview pane:
I’ve always had the impression that writing a shell preview handler was a daunting task involving lots of native code implementing obscure COM interfaces. I’ve also been sure that the managed interop story was very sad, for the following simple (and convincing) reason: Assume that all preview handlers ran inside the explorer.exe (or outlook.exe) process. What if preview handler A was built against CLR v1.1 and preview handler B was built against CLR v2.0? Well, if handler B was loaded first, CLR 2.0 would be loaded into the process and then handler A would happily use it – after all, CLR 2.0 is backwards compatible with CLR 1.1. However, if handler A was loaded first, CLR 1.1 would be loaded into the process, and handler B would not be able to run at all – because there is just one version of the CLR that can be loaded into a given process. (This last remark will cease to be true as of CLR 4.0, but preview handlers have been there for a couple of years now.)
However, this was a myth. I will be the first to admit that I didn’t do my homework. Preview handlers always run out of process, by default in the surrogate host called Prevhost.exe (these handlers won’t be compatible with Outlook 2007 on Windows XP). How is this any better than running all preview handlers in the Explorer.exe process?
Well, it’s possible to specify that your preview handler should be launched in a separate instance of Prevhost.exe instead of sharing the default one. This effectively gives you the tools to separate managed preview handlers into different processes, e.g. according to the version of the CLR they were built against.
Still, writing a managed preview handler wouldn’t be trivial – there are several COM interfaces to implement, and some registry keys to set so that your handler is recognized by the shell.
Fortunately, Stephen Toub’s amazing article from MSDN Magazine titled “Writing Your Own Preview Handlers” provides a managed framework for developing preview handlers. Writing a managed preview handler using that framework is as easy as:
- Deriving a class from StreamBasedPreviewHandler or FileBasedPreviewHandler
- Implementing the CreatePreviewHandlerControl method
- Deriving a class from StreamBasedPreviewHandlerControl or FileBasedPreviewHandlerControl
The rest is the details of drawing a Windows Forms representation of your file type (and of course you can integrate WPF inside using the ElementHost wrapper control).
If you never got the chance to experiment with the Windows Shell, writing a managed preview handler is an incredibly easy opportunity to get started. If there’s any file type that you wished there was a preview handler for, you can finally write one yourself!
In the first part, we looked at a scenario where fiddling with the in-memory field controlling a custom cache size and then externally triggering a garbage collection gave us a likely culprit for a high memory scenario.
After having disentangled the first problem, we faced a completely different issue. One of the apparently-leaking processes didn’t have that many objects in its managed heap. Inspecting the GC performance counters indicated that there while memory utilization was around 900MB, only 150MB or so of managed objects were actually on the managed heap.
We ran the !eeheap –gc WinDbg command to take a look at segment sizes, and noticed the following: In each heap (this is an 8-core system running server GC, so 8 heaps) there was just a single ephemeral segment (gen0 and gen1) but it was HUGE. And I’m not talking about reserved virtual memory – I’m talking about >100MB of committed memory in a segment, with only about 10-20MB of it actually used to store managed objects.
It was as if the memory from these segments was never being given back to the OS… Which sounds remotely familiar, doesn’t it? Well, it did to me – that’s exactly what the VM hoarding feature is all about. On a 64-bit system, it makes little sense to give virtual address space back to the OS, and it’s fairly likely that you would want to specify the STARTUP_HOARD_GC_VM flag if you’re writing your own CLR host and binding to the CLR.
But we’re not writing a CLR host here, now are we? Well, we aren’t, but ASP.NET is a custom host in and of its own. It already triggers server GC for you even though you didn’t ask it to, so why wouldn’t it specify some more of that magic sauce in the hope of maximizing performance?
Well, it might, but how do we find out?
At this point, we were a little blocked but fortunately, after crawling the Rotor (SSCLI) sources, Dima Zurbalev found an undocumented global flag called g_IGCHoardVM, which apparently indicated whether VM hoarding was on.
Luckily for us, this global flag was exported from mscorwks.dll, so we could simply inspect it at runtime with the WinDbg db command and see that it was set to 1! To confirm, we inspected its value in a simple console application that didn’t request VM hoarding, and it was set to 0 – so we could reach a reasonable conclusion that ASP.NET was indeed requesting VM hoarding.
What this basically means is that memory is never reclaimed by the OS – your application’s memory usage is dominated by its peak memory usage. This is not a memory leak – if memory was running out, these segments would likely be freed – but we couldn’t tell if it were the case or not because the customer was never able to show us a system that actually crashed with an OOM condition.
Paradoxically, our recommendation in this case was to remove the automatic high-memory alert and recycling system they had in place (at least for one of their servers), and see if it crashed with an OOM condition. If it did, we would know for sure that VM hoarding was not the only culprit here.
I’ve just noticed a post by Guy Kolbis discussing a possible solution for deadlocks – ensuring that all locks are taken with a timeout. To do so, Guy cites the TimedLock struct, originally introduced by Ian Griffiths.
The general idea is that instead of using a standard lock{…} block, you wrap your critical section in the following statement:
using (TimedLock.Lock(...))
{
}
Ian even introduces a refinement of this idea by including a sentinel reference type in the struct, in Debug builds, so that if the lock isn’t explicitly freed, the sentinel’s finalizer will be called and raise an assertion – so that the developer is immediately notified that there was a lock she forgot to release.
Unfortunately, there is a subtle yet fatal flaw with this approach. Consider the proposed constructor of TimedLock:
public static TimedLock Lock (object o, TimeSpan timeout)
{
TimedLock tl = new TimedLock (o);
if (!Monitor.TryEnter (o, timeout))
{
#if DEBUG
System.GC.SuppressFinalize(tl.leakDetector);
#endif
throw new LockTimeoutException ();
}
return tl;
Can you spot the problem?
Here’s a hint – what happens if an exception occurs after the lock was successfully acquired, but before the method returns?
Let’s step back for a second – how can an exception possibly occur between these two lines? After all, if the TryEnter method returned successfully, there are no other operations in this method that could possibly introduce an exception. Or maybe there are?
Of course there are! If your thread is asynchronously aborted, e.g. using the Thread.Abort method, or as a result of an asynchronous exception such as OutOfMemoryException, then the exception can be potentially introduced between any two instructions in your code, including the most innocuous ones.
And what happens then? Well, because the exception occurs before the method has returned, the using statement does not actually dispose the lock because the reference to the lock is never assigned in the caller’s code. If you’re running in Debug, you would probably catch it thanks to the sentinel reference type – but if this happens to you in a production scenario, you have just leaked an acquired lock, which is never going to be released.
You can easily emulate this by putting a trace statement in the Dispose method, introducing a short sleep into the static Lock method (between the two problematic lines we discussed above), and then inducing an asynchronous exception from a different thread while the Lock method is executing – e.g. by aborting that thread.
So essentially, while the TimedLock apparently solves the problem of deadlocks, it does so by reintroducing potential deadlocks. Not much gained here.
Nitpicker’s Corner
You, Dear Reader, might have a completely legitimate question. Namely: Why is the lock keyword immune from this problem?
As you probably know, a lock statement is compiled to a call to Monitor.Enter, and then a try…finally block. In the finally block, Monitor.Exit is called.
The JIT code generation for both x86 and x64 ensures that a thread abort cannot occur between a Monitor.Enter call and a try block that immediately follows it. This always worked that way for x86/Release (because there were no instructions between the call and the subsequent instruction within the try block), and was actually fixed in VS2008 for x64/Release. (What we have in our case is a try block that is quite a few instructions, including a function call return, away from the Monitor.Enter call.)
A few days ago, I was visiting a customer with high-memory scenarios in a 64-bit ASP.NET application. I’m saying “scenarios” because we still don’t know for sure what the problem was, and because we’re pretty confident that there’s more than one underlying cause.
Even though there are no conclusive results yet, I wanted to share with you some of the things we did because they are interesting on their own behalf.
What we were facing was a set of 4 w3wp.exe processes that were consistently increasing in memory consumption over a period of several hours, starting from approximately 300MB and climbing to over 1GB of memory. Because this was a 64-bit system, I was not concerned with memory fragmentation, but overall system physical memory was starting to run out. To save themselves from the dangers of paging, the company had an alert system in place to automatically notify (and restart) when physical memory utilization was around 95%.
Armed with this knowledge, we started looking at memory dumps, which of course contained millions of apparently interesting objects, all rooted at or related to a distributed cache component that the company was using. The architecture of this distributed cache allows for two tiers of caching on each machine: A service that keeps the object data in serialized form, receiving notifications and objects from other nodes in the distributed cluster, and a first-tier managed cache inside the ASP.NET application which keeps the object data non-serialized, after having passed it to the service in serialized form.
Looking at the cache configuration, I noticed that it was set to contain a maximum of 100MB of objects, and a timeout value to ensure that objects were not retained indefinitely. Nonetheless, the application’s memory utilization suggested that many more objects were actually stored in the cache than reported.
Fortunately, the first-tier cache is written in .NET, so I was able to take a look at the source code with Reflector. After inspecting the code for a while, I wondered how the cache was even able to tell the size of the managed object (to keep the cache from exceeding the 100MB hard limit). Apparently, what happened at each add-to-cache operation was approximately the following:
- Serialize the object using BinaryFormatter
- Send the serialized representation to the service, including the timeout parameter
- If the current cache size + the size of the serialized object exceeds the cache maximum, remove objects from the cache until the limits are satisfied
- Store a local reference to the object
There are two fundamental things wrong here:
- There is no notion of timeout on the ASP.NET first-tier side – objects are retained indefinitely
- The serialized size of an object is not necessarily the same as its real size in memory!
This second problem is a very likely cause of the non-deterministic memory consumption. If the cache receives an object that takes 4K of memory but due to sheer luck can be serialized to 1K of memory, the cache retains the 4K of memory but reports only 1K as being in use.
We tried several types of objects and there was a particular category of objects that was being serialized to a much smaller form than the original – Hashtables. Since the cache was primarily used to store session objects, which are hardly anything more than a Hashtable instance, we had a fairly good grip on this one.
To verify, I wanted to change the cache configuration to store at most 1K of objects and see if the memory usage will go down. However, the leak took several hours to accumulate, and we wanted results right away. To do so, I attached WinDbg to a live process, used !dumpheap and !do to find the cache configuration object (which, fortunately, was queried on every add-to-cache operation), and used the w command to write the 1K value to memory (instead of the 100MB that was there).
After a few minutes, nothing happened yet – memory was still high and wouldn’t go down. After inspecting the GC performance counters, I realized that no Gen2 garbage collections were occurring, so there was no opportunity for memory to be reclaimed. Because this was a 64-bit system, and because lots of memory was still available, the GC didn’t feel that a collection was necessary, and rightfully so.
So now we were left with the question – how can we trigger a garbage collection even though there was no need for garbage collection? After exploring interesting ideas like injecting code into the process (with e.g. CreateRemoteThread) to trigger a GC, I figured that we might as well consume all available physical memory. It is known that the CLR listens for the low physical memory status notification, and triggers a GC when this notification occurs.
I wrote a simple console app that allocated lots and lots of physical memory, let it run for a few seconds – and voila, a garbage collection in our process occurred. Most of the memory has been reclaimed, showing that the first-tier cache was the most likely culprit.
The Windows 7 taskbar can cause lots of initial grief for people used to the Quick Launch toolbar. After all, you do a clean install of Windows 7 and there is no Quick Launch to be found. (I personally prefer the Vista start menu search box, but that’s just me.)
If you’re one of these people, then before you go and bring Quick Launch back from the dead, why don’t you try using the taskbar itself as your launch surface? By pinning any shortcut to the taskbar, you can effectively create a unified launch and control surface for your frequently used and currently running applications.
In case you’re desperately missing the WinKey+1, WinKey+2 etc. combinations, they are still there! If you have the default Windows 7 taskbar, do WinKey+1 and you should have a new Internet Explorer window pop up.
While we’re at it, why don’t you play around with the other Windows 7 keyboard shortcuts, which promise countless hours of joy and productivity?
My personal favorites:
- WinKey+Up, Down, Left, Right to toggle snapping, maximize and minimize
- WinKey+Shift+Left, Right to move an app between monitors
- WinKey+T to see the thumbnails of a taskbar button (and keep tapping WinKey+T to toggle through all taskbar buttons)
- WinKey+Space to peek at the desktop
I can’t delay it any further: In the “most exciting feature of the Windows 7 desktop” competition, the new taskbar comes first by a margin.
At the first PDC session dedicated to the Windows 7 desktop, Chaitanya Sareen brought us all back to the future with this screenshot of Windows 1.0:
Do you see the origins of the new Windows 7 taskbar? It’s amusing to say it, but the Windows 1.0 taskbar bears more resemblance to the Windows 7 taskbar than the taskbar of Windows Vista does!
The evolution of launch surfaces that has accompanied the past releases of Windows gave us the concepts of Quick Launch, a taskbar area for commonly launched applications; desktop icons for users who like using their desktop as the launch surface; the system tray (a.k.a. the system notification area) for “cool” applications that don’t want to occupy taskbar area, but nag us every now and then with a notification from the bottom-right corner; and of course the Start Menu, and the search text box on the start menu, and …
Well, I don’t know about you, and I haven’t been running around collecting telemetry information the way Microsoft did, but if I were a user to first encounter the Windows Vista desktop, I would be kind of … confused.
So what’s really the message of this new taskbar? It’s not just another incarnation of the same user interface – it’s in fact a revolution of launch surfaces.
Running applications, multiple instances of running applications, pinned programs for easier launch access – these concepts are all consolidated into the new taskbar. The Quick Launch is dead; the notification area is (almost) deprecated and recommended to be out of bounds; large, beautiful taskbar buttons dominate the user experience after the first log on to Windows 7.
And it’s not just about the taskbar buttons! Right click any well-behaved application, and you get a beautiful menu of “things” that are relevant for that application alone:
And these “things” are custom tasks and application destinations, and each app can customize this menu (also called a jump list) to provide the best user experience imaginable.
If you’re writing an app that doesn’t have a well-defined file type, and it seems that you won’t have anything worthy to show in this menu, think again!
Assume you’re writing a patient monitoring application for some healthcare company. Right, there is no .patient file and no .diagnosis extension. But could you have frequent patients in the jump list, so that when a doctor clicks a patient name it will open that patient’s record in the your app? Or you could have the recent patients, which might be interesting at a different time of the day. Or you could have a special category of Important patients, or Infant patients, or …
And tasks? Oh boy, do you have tasks! Open prescription application, close the clinic for the day, manage scheduling and reception hours…
And that’s hardly all. The new taskbar gives us overlay icons, which is a small icon covering a part of your big icon to convey a notification message, instead of cluttering the system notification area with yet another balloon tool tip and yet another icon and yet another application that won’t close:
And you can have a progress bar in the taskbar icon:
And you have thumbnail toolbars, which are little buttons that appear under your application’s thumbnail and give the user even more control over the app – it’s in fact like a remote control with a preview!
And you have thumbnail previews, which can be customized to your will:
And you have a live desktop preview of any window from the thumbnail itself, without the hard window switching and switching back work:
Have I whetted your appetite? Do you desperately want to know how to use the new Windows 7 APIs to integrate your application into this spectacular experience?
You now have a differentiation opportunity like no other. By fully using the functionality of the new taskbar and accommodating your application to take advantage of it, you will shine among the competition and produce software that is consistent with Windows, consistent with the user experience and much easier and more fun to use.
In the rest of this series about the taskbar, we will look into various ways for accomplishing this goal. The Windows 7 SDK, the managed wrappers that we developed for the DPE Windows 7 training, and some common sense design guidelines will lead us through this path.
You might have heard about this feature before: HomeGroups bring easy home networking back into Windows. No more complicated setup steps, transferring settings between computers on USB sticks and praying that the firewall configuration allows Joe’s PC to access Jane’s laptop.
I won’t reiterate everything that has already been said – the above link is an exhaustive explanation of what a homegroup is and how it works. Just wanted to share with you that I am using a homegroup at home between my 3 (and counting) machines running Windows 7, and that the sharing experience couldn’t be any more convenient. The integration into Windows Explorer is seamless, which is what makes it such a great deal:
And the nicest feature of all is that when I’m home, the HomeGroup node is part of the tree and when I’m not, it’s not – the same applies to library locations, the search interface and so on.
Another thing is that a homegroup works even on a domain-joined PC. E.g. my laptop is part of a domain, so other machines in my home can’t see the corporate documents, but the laptop can see outside and use all shared media, documents and information that are part of the homegroup.
This is highly convenient even for me, and I never had too much trouble configuring networking around my home. Imagine what an incredible feature this is for less computer-savvy users.
If you ever encountered application problems in the field, you know how valuable it is to obtain an exact description of what happened from the customer. If you’re into production debugging, you also know that a dump file can be of enormous help, but often it is simply not enough.
Imagine that there is a bug in Microsoft Word which causes it to crash if, after modifying the document properties, changing the default heading style, selecting a line and clicking the ribbon copy command – you click Alt-F4 to close the main window.
How easy do you think this problem would be to reproduce from a crash dump?
For accurate reproductions of what exactly the user did to cause the problem, we often resort to talking to our customers over the phone, instructing them to take a screenshot of the application time and again until the problem occurs.
Fortunately, Window 7 finally gives us a built-in ability to streamline this process – with the Problem Steps Recorder (a.k.a. “Record steps to reproduce a problem”).
Type “problem steps recorder” into the start menu search box, and click on the first result from the Control Panel. You will get something like this:
Now, press Start Record and play around with your system for a short while. Switch between windows, type text, click some buttons. At the end, press Stop Record and choose a destination for the report file, zipped up and ready to be sent to the support personnel.
Inside the zip file you will find an MHTML document, which contains exactly the steps that were performed between the time you clicked Start Record and the time you clicked Stop Record. The specific window, the text, the user actions – it’s all captured in the report. For example:
Problem Step 4: User left click on "Format (menu item)" in "Windows 7 Problem Steps Recorder - Windows Live Writer"
Problem Step 5: User left click on "Align (menu item)"
Oh, and the best thing? It comes equipped with screenshots! Here’s an example from my report (note how the context of the user’s action is highlighted in green):
There’s also a textual representation of all the steps at the bottom of the report, perhaps to facilitate parsing by automatic support and classification programs.
By the way, if you happen to be authoring courses, hands-on labs, demos and other stuff of that ilk, you will LOVE the way Problem Steps Recorder can streamline screenshot generation and demo steps for you. It’s not just a Problem Steps Recorder, you know.
This is huge – not because there was no software that could accomplish this task, but because Problem Steps Recorder is built into Windows 7. It will be there on any customer’s computer. Waiting for you to run it. Waiting for you to receive a concise repro of the problem with your application, to capture a series of demo steps, to explain to someone how a certain task can be accomplished.
At various points in time during the last 3 months I have had lots of different Windows 7 builds installed on various machines. Now that the Beta is out, we can focus solely on the Beta build (a.k.a. build 7000).
The first thing you will notice about the Windows 7 Beta is the incredibly short installation time. Even on the fairly slow 5400RPM 1.8” drive that came with my new Dell Latitude XT, it didn’t take more than 30 minutes from initial boot to the fully configured desktop. This is a huge improvement. (By the way, the upgrade process takes much longer, and I assume that it’s linear in the amount of files and registry keys that you have on the original box. It took me around 1:20 to do upgrades between different Windows 7 builds.)
The next thing you’ll notice about Windows 7 is the new taskbar; but let’s just skip the taskbar for a little while (don’t worry – we will come back to it later). Right now, if you have a Windows 7 machine installed, open Task Manager. Go ahead, open it. Here’s mine:
Have you EVER seen a Windows operating system that is actually idle when idle? Well, Windows 7 certainly is. Even though it requires great cooperation from applications and drivers, it is to some extent also the responsibility of the operating system to minimize idle CPU activity. And 0% is something I would certainly call minimized. By the way, when I took the screenshot above I had Outlook, Internet Explorer, Live Writer, Messenger, Visual Studio and several other apps open. (You would also notice memory consumption being significantly lower than that of Vista – with all these applications open, my primary work laptop drifts regularly between 1.4 to 1.8 GB of memory usage.)
Now for my installations of Windows 7…
I have Windows 7 installed on my work laptop as dual boot with Vista. I only ever boot into Vista when I have to check whether some code I wrote works fine on Vista, and to make sure the Vista installation is regularly patched. I feel there’s absolutely no reason to go back – and remember, we’re talking about a Beta.
I have Windows 7 installed on the Dell Latitude XT laptop which I got to work on the multi-touch features of Windows 7 (it comes with the N-Trig multi-touch hardware). Windows itself works flawlessly, including all drivers for esoteric hardware such as the fingerprint sensor. The N-Trig drivers are not that flawless, but with multi-touch disabled the system is as stable as theoretically possible.
I have Windows 7 installed on my HP Touchsmart, an amazing all-in-one 25.5” dual-touch computer. The HP dual-touch hardware works great from Windows 7, and I can’t wait for more applications to start taking advantage of multi-touch gestures. The Windows 7 Media Center, which I use a lot on this box, is the first edition of Media Center that I find responsive enough for my needs. I’m not sure yet if it’s thanks to the OS or the application itself, but for now I couldn’t care less :-)
Finally, I have Windows 7 installed in a virtual machine with 512MB of memory. It boots, starts up and works perfectly within these constraints – and I haven’t noticed any substantial improvement in e.g. startup performance when I increased the amount of memory given to the guest OS. (Of course I am not using this VM with 7 instances of Visual Studio, but for quick demos it works just great.)
It is such a joy to write this post after almost 3 months of blogging silence! (It also gives me an opportunity to apologize to you, Dear Reader, for this silence.)
During the past 3 months I have been busy preparing training materials, hands-on labs and .NET wrappers for Windows 7. This was a collaboration between Sela, including yours truly as well as Alon Fliess and Dima Zurbalev, and the Client DPE team at Microsoft (Redmond).
The result of this process is a comprehensive course with demos, hands-on labs, detailed presentations and lots of .NET sample code. As I write this, Alon is making his way to Redmond to teach this course for the very first time.
Just to give you an idea what materials we were preparing:
- Windows 7 Taskbar
- Sensor and Location Platform
- Multi-Touch
- Shell Libraries
- Scenic Ribbon and Animation
- Instrumentation and Diagnostics
- Direct3D, Direct2D and DirectWrite
- Trigger-Start Services
- Application Compatibility
- Energy Efficiency
- …and lots of other topics
This is also a great opportunity to thank all the parties involved for making this happen: first and foremost, Yochay Kiriaty of Microsoft who initiated this amazing process, Alon Fliess and Dima Zurbalev for pulling of an amazing amount of work in so little time, and of course Sela CEO David Bassa for clearing up the way for us to work on this project almost full-time and attain these incredible results.
Back to Windows 7 though.
If you have been following the blogosphere, you know that there’s lots of interesting things to say about the Windows 7 Beta, which was just released a short while ago. I hold my fingers crossed that I will have time to share with you the exciting new stuff that is part of this release.
In the meantime, let me just tell you this: I have been running various Windows 7 builds for over 3 months now, on nearly all my laptops and PCs. It’s not very easy for an operating system to grab my attention – but this one, by far, is the best operating system I have ever used.