October 2010 - Posts
I gave a couple of examples in my previous post which demonstrate, IMHO, how async methods can simplify existing approaches to asynchrony in C# programs. Both examples were fairly simple in that they didn’t involve cancellation, progress reporting, UI thread dispatching, or exception handling.
I’m happy to report that accomplishing these objectives is as easy as doing nothing else than what we’ve already seen.
This is a result of a fairly sophisticated C# rewriting process that the compiler applies to async methods, not unlike the rewriting process applied to iterator blocks (“yield return” and “yield break”). For example, the following async method:
static async void M() {
S(); //synchronous part
await T(); //queue continuation
C(); //continuation
}
Is compiled to the following C# code, according to Reflector:
private static void M()
{
<M>d__2f d__f = new <M>d__2f(0);
d__f.MoveNextDelegate = new Action(d__f.MoveNext);
d__f.$builder = VoidAsyncMethodBuilder.Create();
d__f.MoveNext();
}
[CompilerGenerated]
private sealed class <M>d__2f
{
//…snipped for brevity
public void MoveNext()
{
try
{
this.$__doFinallyBodies = true;
if (this.<>1__state != 1)
{
if (this.<>1__state == -1)
{
return;
}
Program.S();
this.<a1>t__$await30 =
Program.T().GetAwaiter();
this.<>1__state = 1;
this.$__doFinallyBodies = false;
if (this.<a1>t__$await30.BeginAwait(
this.MoveNextDelegate))
{
return;
}
this.$__doFinallyBodies = true;
}
this.<>1__state = 0;
this.<a1>t__$await30.EndAwait();
Program.C();
this.<>1__state = -1;
this.$builder.SetCompleted();
}
catch (Exception)
{
this.<>1__state = -1;
this.$builder.SetCompleted();
throw;
}
}
}
This rewriting process takes into consideration also exception handling. In other words, your async methods can wrap code in try…catch blocks just like their synchronous counterparts; exceptions that arise in one of the awaited tasks, in the synchronous section of the code, or in one of the continuations, are all propagated into the respective exception handling blocks. For example:
async void WithExceptions()
{
try
{
//An exception that arises here (e.g. in the
//synchronous part of DownloadDataTaskAsync),
//or in the continuation, will be caught by
//the catch block below.
var t = new WebClient().DownloadDataTaskAsync(
"http://www.microsoft.com");
if (t !=
await TaskEx.WhenAny(t, TaskEx.Delay(100)))
{
throw new TimeoutException();
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
Similarly, cancellation takes an approach familiar from .NET 4.0 APIs such as Parallel.For or Task.StartNew. You can pass a CancellationToken into any async method that accepts one, and that async method can rendezvous with the token by using CancellationToken.ThrowIfCancellationRequested or any other method.
Similarly, while you’re using cancellation, you’ll notice a new extension method, CancellationTokenSource.CancelAfter. This is a really simple method to write, and it automagically cancels the operation after the specified timeout has elapsed. You could implement it with a timer, like the CTP does; or you could:
async void CancelAfter(
this CancellationTokenSource cts, int timeout)
{
await TaskEx.Delay(timeout);
try { cts.Cancel(); }
catch (ObjectDisposedException) { }
}
I’ll leave it to you now to download the Visual Studio Async CTP and its accompanying documentation, to watch Anders Hejlsberg’s PDC session, and to get your own impression whether this is a feature that’s going to make your asynchronous programs easier to write and easier to read.
During today’s PDC session on the future of C# and VB, Anders Hejlsberg announced the availability of the Visual Studio Async CTP, implementing a set of changes to the C# language that support a new pattern for asynchronous programming.
There’s lots to be said about the language support and the framework implementation that enables it, and the CTP is not the final words that will be said. There’s a comprehensive set of samples you can find at the MSDN Code Gallery, and I strongly suggest that you download the CTP and run some of these samples today.
That’s what I did.
| Unlike most features introduced into C# during the last few releases, this one isn’t so easy to get right. In fact, I’m willing to bet that most people will not feel immediately at home with the new syntax—the programming style it enables is not natural unless you’re programming in a functional language. |  [Source]
|
Continuation-Passing Style
To make you more comfortable with the key concept of continuation-passing style (CPS), Eric Lippert wrote a great series of posts culminating in today’s PDC announcement.
CPS means that instead of calling a method f, waiting for it to complete, and then proceeding with additional work g, you call f and pass to it g as a continuation—the code you want to run when f completes its original work. This reverses the way the code is written on the screen, and may be very difficult to read at first:
//Non-CPS
int m1(string s) {…}
float m2(int i) {…}
void m3(float f) {…}
//Normal form
void M() {
m3(m2(m1("Hello")));
}
//Explicit form
void M() {
int i = m1("Hello");
float f = m2(i);
m3(f);
}
//CPS
void m1(string s, Action<int> c1) {…}
void m2(int i, Action<float> c2) {…}
void m3(float f, Action c3) {…}
//Explicit form
void M() {
Action c3 = () => {};//Not really necessary
Action<float> c2 = f => m3(f, c3);
Action<int> c1 = i => m2(i, c2);
m1("Hello", c1);
}
//Normal form
void M() {
m1("Hello", i => m2(i, f => m3(f, () => {})));
}
Async Methods
Async methods, introduced in this CTP, use CPS to specify a continuation that should be executed after an asynchronous operation completes. An async method is executed synchronously on the caller’s thread until the first await statement is encountered. An await statement indicates that you want to queue up a continuation to execute when an asynchronous task completes; the caller’s thread is then freed to do other work, and when the awaited task completes, the rest of the method (following the await statement) executes.
There’s a precise transformation from C# to framework methods that enable the above semantics; the following informal transformation might be helpful in understanding the big picture:
async void foo() {
S;
result = await T;
P;
}
//imprecisely translated to:
void foo() {
S; //synchronous, caller’s thread
//the rest is asynchronous:
when T completes {
result = T.result;
P;
}
}
What are the practical scenarios enabled or simplified by the introduction of this new syntax? Here’s a couple of examples; feel free to consult the “official” samples for more.
UI Responsiveness Made Easy
The simplest pattern for keeping your UI responsive and dealing as little as possible with the headaches of updating the UI from another thread is roughly the following:
- Identify the code X that prepares the operation with UI input; this code might have to execute on the UI thread
- Identify the code Y that may run in the background; typically, it takes the input from X and generates an output
- Identify the code Z that processes the output of Y by displaying results onto the screen; this code might have to execute on the UI thread*
- Implement an async method M:
async void M() {
X;
var result = await TaskEx.Run(() => Y);
dispatch Z to the UI thread if necessary;
}
* The “awaiting” mechanism uses the synchronization context at the moment of “awaiting” to post the continuation Z; this means that in WPF/WinForms applications, where a synchronization context is present, there’s no need to worry about dispatching Z appropriately. When there’s no synchronization context, TaskScheduler.Default is used.
Here’s a trivial example of this in a console application:
async void CompareDataLength(string url1, string url2)
{
var t1 = TaskEx.Run(
() => new WebClient().DownloadData(url1));
var t2 = TaskEx.Run(
() => new WebClient().DownloadData(url2));
var results = await TaskEx.WhenAll(t1, t2);
//This is Z, but we don’t need to dispatch anything
//as we can write to the console from any thread.
if (results[0].Length >= results[1].Length)
Console.WriteLine(url1 + " has shorter data.");
else
Console.WriteLine(url2 + " has shorter data.");
}
void CompareDataLengthDemo()
{
//This is X
Console.WriteLine("I will process input now, " +
"even though there's BG work.");
//This is Y
CompareDataLength(
"http://www.google.com", "http://www.bing.com");
//This is to demonstrate that the UI remains responsive
string input;
while ((input = Console.ReadLine()) != "quit")
{
Console.WriteLine("Say 'quit' if you want out.");
}
}
“Workflow Parallel” Processing Style
If you ever used Workflow Foundation, you might have encountered the Parallel activity, an activity that appears to run code in parallel even though there’s just one thread executing the workflow. A typical use for a parallel flow is when you call a set of external components asynchronously and process their responses as they arrive.
The following method demonstrates a parallel call to multiple services. While waiting for the results, we also wait for the delay task, which simply sleeps for a certain amount of time. If this task completes, execution times out.
async void QueryServices()
{
string[] items = { "chair", "banana", "coat" };
var queries =
from item in items
let temp = item
select TaskEx.Run(() => DoItemQuery(temp));
Task<int> delay;
const int TIMEOUT = 2500;
queries = queries.Concat(new[] { delay = TaskEx.Run(
() => { Thread.Sleep(TIMEOUT); return 0; }) });
for (int i = 0; i < items.Length; ++i)
{
var completed = await TaskEx.WhenAny(queries);
if (delay == completed)
{
Console.WriteLine("Timed out.");
break;
}
else
{
int result = await completed;
Console.WriteLine("Result: " + result);
}
}
}
What Else?
There’s lots more to be said about exception propagation, built-in cancellation support, progress reporting, TPL dataflow (an agents library), and other areas where the TPL simply has been extended to make working with tasks and async methods easier. Let’s leave this to a future post.
In the previous installment we saw the core of a lexical analyzer, a module that generates from a stream of characters a set of tokens for symbols, identifiers, keywords, integer constants, and string constants. Today, we move to parsing.
The parser’s job is to give semantic structure to the syntactic tokens bestowed upon it by the lexical analyzer. There are, as always, automatic tools like yacc that create from a BNF grammar a program that parses tokens in a certain language. However, it is often more efficient and certainly more educational to write a parser by hand.
We’ll be dealing with a very special kind of parser—a recursive descent predictive parser. As you might remember from the last installment, Jack is almost an LL(1) language, meaning that a single look-ahead token is sufficient for parsing almost all Jack statements and expressions. (This predictive nature is a property of deterministic finite automata.)
When you look at a Jack program fragment (such as the following one, taken from the previous installment), you probably see a certain structure, but translating that structure into a set of parsing procedures is usually non-trivial without some sort of grammar representation.
let prime = true;
while (i < numbers[j] & prime) {
if (numbers[j] % i = 0) {
do System.printInt(numbers[j]);
do System.print(" is composite.");
do System.println();
let prime = false;
}
let i = i + 1;
}
One such convenient representation is Backus-Naur Form (BNF).
If you’re familiar with regular expressions, you should have no difficulty understanding the following simplified BNF definitions of some of Jack’s grammar:
stmt ::= while-stmt | if-stmt | let-stmt
while-stmt ::= while '(' expr ')' { stmt* }
if-stmt ::= if '(' expr ')' { stmt* }
let-stmt ::= let var ( [ expr ] )? = expr ;
var ::= identifier
identifier ::= letter_ ( letter_ | digit )*
letter_ ::= A | B | … | Z | a | b | … | z | _
digit ::= 0 | 1 | … | 9
The entire Jack grammar can be expressed in less than a single printed page of BNF definitions. I hope you’re convinced by now that BNF is a concise form of expression; it remains for me to convince you that it’s also a convenient form.
Let’s take a simple BNF grammar for summation expressions, where the terms are separated by + and there are no parentheses:
summation ::= term ( '+' term )* ;
term ::= digit+
Assuming that the tokenizer hands us the terms (so that we don’t have to parse individual digits), the parser almost writes itself from the BNF:
procedure ParseSummation
ParseTerm()
while LookaheadToken <> ";"
Match("+")
ParseTerm()
//placeholder (1)
end while
end procedure
procedure ParseTerm
term := NextToken()
//placeholder (2)
end procedure
One thing this parser could do is convert infix summation expressions to postfix summation expressions. (Later, postfix will be an excellent choice for actually compiling a Jack expression to assembly language.) To accomplish that, we need to replace the first placeholder with “print +” and the second placeholder with “print term”.
We can parse more complex expressions in a very similar fashion; in the next installment we’ll deal with the full BNF form of a Jack expression.
I recently stumbled upon the All-In-One Code Framework project on CodePlex. It’s a very impressive collection of samples in C#, VB.NET, and C++, and most of the samples I’ve seen are of superb quality.
One thing that’s cool about this project is that they publish their own set of coding standards. It’s a very elaborate 87-page document written in the “Framework Design Guidelines style” with “DO” and “DO NOT” items sprinkled liberally throughout the guidelines. It’s relevant for C# and C++ developers alike.
Anyway, here are some nuggets of wisdom I gathered from the document:
- You should use braces around single line conditionals. Doing this makes it easier to add code to these conditionals in the future and avoids ambiguities should the tabbing of the file become disturbed. [Italics mine. –Sasha]
- Do not use Enum.IsDefined for enum range checks in .NET. There are really two problems with Enum.IsDefined. First it loads reflection and a bunch of cold type metadata, making it a surprisingly expensive call. Second, there is a versioning issue here. [Instead, use > and < to test for the enum value falling within the desired range. –Sasha]
- Do use ARRAYSIZE() as the preferred way to get the size of an array. ARRAYSIZE() is declared in a way that produces an error if it is used on a non-array type, resulting in error C2784. For anonymous types you need to use the less safe _ARRAYSIZE() macro. ARRAYSIZE() should be used instead of RTL_NUMBER_OF(), _countof(), NUMBER_OF(), etc.
- Do use "= {}" to zero array memory. The compiler optimizer does better with "= {}" than "= {0}" and ZeroMemory, so "= {}" is preferred.
- Do not overload operator&&, operator|| or operator,. Unlike the built-in &&, || or , operators the overloaded versions cannot be short-circuited, so the resulting behavior of using these operators typically isn’t what was expected. [This paragraph relates to the C++ version of these operators. –Sasha]
- Do use IFACEMTHODIMP and IFACEMTHODIMP_ for method declarations in COM interfaces. These macros have replaced the usage of STDMETHODIMP and STDMETHOD as they add the __override SAL annotation.
- Do declare private destructors (or protected if you expect people to derive from you) for classes that implement COM objects that are allocated on the heap. This avoids clients mistakenly calling “delete pObj”, something that should only happen when the ref count of the object goes to zero.
- Do implement IEquatable<T> on value types. The Object.Equals method on value types causes boxing and its default implementation is not very efficient, as it uses reflection. IEquatable<T>.Equals can have much better performance and can be implemented such that it will not cause boxing.
- Do not provide set-only properties. If the property getter cannot be provided, use a method to implement the functionality instead. The method name should begin with Set followed by what would have been the property name.
- Do use member overloading rather than defining members with default arguments. Default arguments are not CLS-compliant and cannot be used from some languages. There is also a versioning issue in members with default arguments. Imagine version 1 of a method that sets an optional parameter to 123. When compiling code that calls this method without specifying the optional parameter, the compiler will embed the default value (123) into the code at the call site. Now, if version 2 of the method changes the optional parameter to 863, then, if the calling code is not recompiled, it will call version 2 of the method passing in 123 (version 1’s default, not version 2’s default).
- An internal constructor can be used to limit concrete implementations of the abstract class to the assembly defining the class.
I’m going to write a compiler for a simple language. The compiler will be written in C#, and will have multiple back ends. The first back end will compile the source code to C, and use cl.exe (the Visual C++ compiler) to produce an executable binary.
But first, a minor digression.
Over my blogging years, I developed this tendency of abandoning blog post series just prior to their final installment. I abandoned the unit testing series, the primality testing series, and many other “series”.
Therefore, I’m not going to call this thing a “series”. I might be able to write another post on the same subject, or I might not; if I don’t, I’ll post the whole lump of source code here and have you decide if it’s worth continuing on your own.
With that said, let’s start by introducing the language for which we’ll write a compiler. It’s called Jack, and I haven’t made it up—it’s a teaching language used in the book The Elements of Computer Systems by Noam Nissan and Shimon Shocken, with some minor modifications I introduced. The language is designed to make lexical analysis, parsing, and code generation as easy as possible. (Indeed, the HUJI course From NAND to Tetris covers compiler construction in two lessons, and students complete a working Jack compiler—to an intermediate VM representation—in slightly less than three weeks.)
Next, the obligatory “Hello World” program in Jack:
class Main {
function void main() {
do System.print("Hello World!");
do System.println();
}
}
And now a more realistic example that demonstrates some of Jack’s coding constructs:
class Main {
function Array initNumbers() {
var Array numbers;
let numbers = Array.new(5);
let numbers[0] = 13;
let numbers[1] = 14;
let numbers[2] = 41;
let numbers[3] = 97;
let numbers[4] = 101;
return numbers;
}
function void main() {
var Array numbers;
var int i, j;
var boolean prime;
let numbers = Main.initNumbers();
let j = 0;
while (j < 5) {
let i = 2;
let prime = true;
while (i < numbers[j] & prime) {
if (numbers[j] % i = 0) {
do System.printInt(numbers[j]);
do System.print(" is composite.");
do System.println();
let prime = false;
}
let i = i + 1;
}
if (prime) {
do System.printInt(numbers[j]);
do System.print(" is prime.");
do System.println();
}
let j = j + 1;
}
}
}
This program’s output is:
C:\JackCompiler>JackCompiler.exe HelloWorld.jack
C:\JackCompiler>out
13 is prime.
14 is composite.
41 is prime.
97 is prime.
101 is prime.
Assuming that we’re not interested in extraneous formalism, we can go ahead and think about the first part of the compiler—the lexical analyzer, or the tokenizer. The structure of a compiler is well-illustrated by the following diagram [source]:
Before we attach semantic meaning to the language constructs, we have to get away with such details as skipping unnecessary whitespace, recognizing legal identifiers, separating symbols from keywords, and so on. This is the purpose of the lexical analyzer, which takes an input stream of characters and generates from it a stream of tokens, elements that can be processed by the parser. Sometimes the parser constructs a parse tree (abstract syntax tree) or any other intermediate representation of the source code; at other times, the parser directly instructs the compiler back-end (or code generator) to synthesize the executable program.
Normally, you wouldn’t write the lexical analyzer by hand. Instead, you provide a tool such as flex with a list of regular expressions and rules, and obtain from it a working program capable of generating tokens. For example, the following regular expression recognizes all legal Jack identifiers:
[_A-Za-z][_A-Za-z0-9]*
However, for didactic reasons, we will be rolling by hand our own lexical analyzer. It’s not a very challenging task, too—dealing with comments and extraneous whitespace is probably the hardest part.
The following is the primary method of our lexical analyzer. (The rest of its implementation was omitted for brevity.)
public void Advance()
{
EatWhitespace();
if (IsAtEnd)
{
_done = true;
return;
}
char nextChar = NextChar();
if (Syntax.IsSymbol(nextChar))
{
//This token is going to be a symbol. There are
//three special look-ahead cases for '<=', '>=',
//and '!='.
if ((new[] { '<', '>', '!' }.Contains(nextChar))
&& LookAhead() == '=')
{
NextChar();//Eat the '='
_currentToken = new Token(
TokenType.Symbol, nextChar + "=");
}
else
{
_currentToken = new Token(
TokenType.Symbol, nextChar.ToString());
}
}
else if (Syntax.IsNumber(nextChar))
{
//This token is going to be an integer constant.
string intConst = nextChar.ToString();
intConst += EatWhile(Syntax.IsNumber);
int result;
if (!int.TryParse(intConst, out result))
{
throw new CompilationException(
"Int const must be in range [0,2147483648), " +
"but got: " + intConst, _currentLine);
}
_currentToken = new Token(
TokenType.IntConst, intConst);
}
else if (Syntax.IsCharOrdinalStart(nextChar))
{
char marker = NextChar();
if (marker == '\\')
{
string code = EatWhile(Syntax.IsNumber);
if (code.Length != 3)
{
throw new CompilationException(
"Expected: \\nnn where n are decimal digits",
_currentLine);
}
int value = int.Parse(code);
if (value >= 256)
{
throw new CompilationException(
"Character ordinal is out of range [0,255]",
_currentLine);
}
_currentToken = new Token(
TokenType.IntConst, value.ToString());
}
else
{
_currentToken = new Token(
TokenType.IntConst, ((int)marker).ToString());
}
NextChar();//Swallow the end of the character ordinal
}
else if (Syntax.IsStringConstantStart(nextChar))
{
//This token is going to be a string constant.
string strConst = EatWhile(
c => !Syntax.IsStringConstantStart(c));
NextChar();//Swallow the end of the string constant
_currentToken = new Token(
TokenType.StrConst, strConst);
}
else if (Syntax.IsStartOfKeywordOrIdent(nextChar))
{
string keywordOrIdent = nextChar.ToString();
keywordOrIdent += EatWhile(
Syntax.IsPartOfKeywordOrIdent);
if (Syntax.IsKeyword(keywordOrIdent))
{
_currentToken = new Token(
TokenType.Keyword, keywordOrIdent);
}
else
{
_currentToken = new Token(
TokenType.Ident, keywordOrIdent);
}
}
else
{
throw new CompilationException(
"Unexpected character: " + nextChar, _currentLine);
}
}
There are five interesting cases here from which five different token types can be generated:
- A symbol [TokenType.Symbol], which may contain two characters—explaining the need for additional look-ahead with ‘<’, ‘>’, and ‘!’. Note that the additional look-ahead may fail if the symbol is placed at the end of the file, but this is not a legal language construct, anyway.
- A numeric constant [TokenType.IntConst]—we currently allow only integer constants, as the language doesn’t have floating-point support.
- A character ordinal constant such as ‘H’ or ‘\032’—these are translated to numeric constants as in #2.
- A literal string constant [TokenType.StrConst] such as “Hello World”—note that ‘”’ is not a legal character within a literal string constant. We leave it for now as a language limitation.
- A keyword or an identifier [TokenType.Keyword or TokenType.Ident], matching the previously shown regular expression.
This lexical analyzer is rather “dumb”—it does not record identifier information anywhere, and it doesn’t provide access to anything but the current token. It turns out that we don’t need anything else for the current Jack syntax—formally speaking, it is almost an LL(1) language, i.e. most of its language constructs can be parsed with only one look-ahead token. The single LL(2) exception is subroutine calls within expressions, and we’ll craft a special case in the parser to work around this limitation.
For the “Hello World” program above, this lexical analyzer will produce the following sequence of tokens:
<keyword, class> <ident, Main> <symbol, {>
<keyword, function> <keyword, void> <ident, main>
<symbol, (> <symbol, )> <keyword, do>
<ident, System> <symbol, .> <ident, print>
<symbol, (> <symbol, "> <strconst, Hello World!>
<symbol, )> <symbol, ;> …
Next time, some parsing.
Some references if you want to keep reading while I’m writing the subsequent parts:
This post was inspired by a debugging war story Dima Zurbalev told me a couple of days ago. All the credit for finding the bug and describing the diagnostic process belongs to Dima. (This isn’t the first time—see Garbage Collection Thread Suspension Delay and Improving Cold Startup Performance for two additional examples of Dima’s work.)
Anyway, the case in point is a WinForms application that stops responding (I reproduced this in a simple scenario, but originally the problem was obscured by additional details). After attaching WinDbg to the process and loading SOS, the managed thread inspection shows three threads:
0:004> !Threads
[…edited for brevity…]
0 1 528 STA
2 2 f14 MTA (Finalizer)
4 3 1f74 MTA (Threadpool Worker)
And the only thread with a current managed call stack is the main thread:
0:004> ~0s; !CLRStack
[…edited for brevity…]
OS Thread Id: 0x528 (0)
ESP EIP
0024e7f0 771b723b [NDirectMethodFrameStandalone: 0024e7f0] System.Windows.Forms.SafeNativeMethods.GetWindowTextLength(System.Runtime.InteropServices.HandleRef)
0024e808 7089cb58 System.Windows.Forms.Control.get_WindowText()
0024e838 7089c8dc System.Windows.Forms.Form.get_WindowText()
0024e844 7089c82b System.Windows.Forms.Control.get_Text()
0024e850 7089c7f5 System.Windows.Forms.Form.get_Text()
0024e854 7089de91 System.Windows.Forms.Control.set_Text(System.String)
0024e868 7089de55 System.Windows.Forms.Form.set_Text(System.String)
0024e86c 00380522 MyNiceApp.MyMainForm.btnDoIt_Click(System.Object, System.EventArgs)
[…snipped…]
0024ece8 70865911 System.Windows.Forms.Application.Run(System.Windows.Forms.Form)
0024ecfc 003800ae MyNiceApp.Program.Main()
0024ef20 72891b5c [GCFrame: 0024ef20]
From this call stack, it seems that the main thread is stuck in Control.get_WindowText(), an operation that is not supposed to take a significant amount of time. The native method on the top of the call stack is:
0:000> kb
ChildEBP RetAddr Args to Child
0024e754 771c1c01 00060a7c 0000000e 00000000 USER32!NtUserMessageCall+0x15
0024e794 771dffd8 0104b3f0 00000000 00570ce2 USER32!SendMessageWorker+0x5e9
0024e7bc 0027baf6 00060a7c 80b49efb 00000000 USER32!GetWindowTextLengthW+0x40
WARNING: Frame IP not in any known module. Following frames may be wrong.
0024e7d8 7089cb58 02662088 026621d8 0266e2f0 0x27baf6
…all righty, an attempt to send a message to another window to retrieve its text. The window handle is the first parameter of GetWindowTextLengthW, and it’s 00060a7c. Now the question shifts to that window—which thread owns that window? This thread is supposed to answer window messages directed to that window.
Enter our good friend Spy++ that still ships with Visual Studio. In Spy++, a window handle is sufficient to locate the actual window as well as the thread that owns it.
Here’s the find result:
And the thread that owns the window:
Let’s see. The thread ID looks familiar. This is one of the threads from our application!
0:004> !Threads
[…edited for brevity…]
0 1 528 STA
2 2 f14 MTA (Finalizer)
4 3 1f74 MTA (Threadpool Worker) Oh my. The thread pool thread owns this window! What is the thread pool thread doing right now?
0:000> ~4s; k
[…edited for brevity…]
ChildEBP RetAddr
05cff614 759f0816 ntdll!ZwWaitForSingleObject+0x15
05cff680 77501184 KERNELBASE!WaitForSingleObjectEx+0x98
05cff698 728ac661 KERNEL32!WaitForSingleObjectExImplementation+0x75
05cff6dc 728ac597 mscorwks!PEImage::LoadImage+0x1af
05cff72c 728ac5b6 mscorwks!CLREvent::WaitEx+0x117
05cff740 729ee03f mscorwks!CLREvent::Wait+0x17
05cff7c0 729ee4cd mscorwks!ThreadpoolMgr::SafeWait+0x73
05cff824 729d1ec9 mscorwks!ThreadpoolMgr::WorkerThreadStart+0x11c
05cff948 77503677 mscorwks!Thread::intermediateThreadProc+0x49
05cff954 77a59d42 KERNEL32!BaseThreadInitThunk+0xe
05cff994 77a59d15 ntdll!__RtlUserThreadStart+0x70
05cff9ac 00000000 ntdll!_RtlUserThreadStart+0x1b
It seems to be waiting—it’s a thread pool thread, after all, and it doesn’t have any work to do right now. One thing this thread is not doing, however, is pumping window messages—and this is why our attempt to retrieve the window’s text is failing. There is simply no one listening on the other side of the SendMessage call.
Clearly, creating a window from a thread pool thread is a bad idea. But how do you detect this kind of thing? If this were a native Win32 application, we could use Application Verifier. One of the stops under the Threadpool category is “Unclosed window belonged to the current thread”. This is so cool that it’s worth a demo of its own. After properly configuring Application Verifier…
…you would get the following in a debugger if a thread pool thread orphans a window:
=======================================
VERIFIER STOP 00000703: pid 0x1680: Unclosed window belonged to the current thread.
012B132F : Callback function.
00000066 : Context.
03F98A58 : Threadpool Object allocation stack trace, use dps to dump it.
00000000 : Not Used.
threadpool thread (98) having executed Callback (012B132F) has valid hwnd (70fc4: #32770) which could receive messages
=======================================
This verifier stop is continuable.
After debugging it use `go' to continue.
=======================================
(1680.98): Break instruction exception - code 80000003 (first chance)
[…snipped…]
vrfcore!VerifierStopMessageEx+0x4ca:
69853b61 cc int 3
0:001> ln 012B132F
(012b132f) AnMFCApp!ILT+810(?MyCallbackYGKPAXZ)
0:001> dps 03F98A58
03f98a58 629eb309 vfbasics!AVrfpRtlQueueWorkItem+0x2c
03f98a5c 7751ce2a kernel32!QueueUserWorkItem+0x14
03f98a60 012b2ec9 AnMFCApp!CAnMFCAppDlg::OnBnClickedOk+0x39
03f98a64 7868f632 mfc100ud!_AfxDispatchCmdMsg+0xb2 03f98a68 7868fd7a mfc100ud!CCmdTarget::OnCmdMsg+0x2ea 03f98a6c 786e9ac3 mfc100ud!CDialog::OnCmdMsg+0x23 03f98a70 787c6c54 mfc100ud!CWnd::OnCommand+0x174
03f98a74 7844977a mfc100ud!CDialogEx::OnCommand+0x3a
[…snipped…]
Note that the above output also includes the call stack that created the work item that resulted in orphaning the window!
However, in this case we’re dealing with a managed thread pool thread, and the managed thread pool doesn’t use the Win32 thread pool. Therefore, we have to resort to the old method of setting a thread-dependent breakpoint in CreateWindowExW or one of the managed APIs (e.g., Control..ctor). This would give us the exact location in code that creates a window from a thread pool thread, and is left as an exercise to the reader.
Another variety of the same problem (which mimics even more closely the original bug Dima encountered) is a WinForms-initiated cross-thread call from Control.Invoke. In this case, the managed call stack of the (stuck) GUI thread would be similar to the following:
0:000> !CLRStack
OS Thread Id: 0x1e78 (0)
ESP EIP
002eec74 77a400ed [HelperMethodFrame_1OBJ: 002eec74] System.Threading.WaitHandle.WaitOneNative(Microsoft.Win32.SafeHandles.SafeWaitHandle, UInt32, Boolean, Boolean)
002eed20 71ea685f System.Threading.WaitHandle.WaitOne(Int64, Boolean)
002eed3c 71ea6815 System.Threading.WaitHandle.WaitOne(Int32, Boolean)
002eed50 70dc1a51 System.Windows.Forms.Control.WaitForWaitHandle(System.Threading.WaitHandle)
002eed68 708a4a20 System.Windows.Forms.Control.MarshaledInvoke(System.Windows.Forms.Control, System.Delegate, System.Object[], Boolean)
002eee0c 70dc33d0 System.Windows.Forms.Control.Invoke(System.Delegate, System.Object[])
002eee40 70dc3367 System.Windows.Forms.Control.Invoke(System.Delegate)
002eee44 00660595 MyNiceApp.MyMainForm.btnDoIt_Click(System.Object, System.EventArgs)
[…snipped…]
002ef2ec 006600ae MyNiceApp.Program.Main()
002ef514 72891b5c [GCFrame: 002ef514]
As you see, the implementation of Control.Invoke waits for an event. This event, as it is evident from the method’s implementation, is signaled when the target of the Invoke call completes the invocation on another thread. However, it is now more difficult to determine which window we are trying to call into and which thread owns it, as there is no SendMessage on the unmanaged call stack.
Fortunately, we have the source code of the MyNiceApp.MyMainForm.btnDoIt_Click method:
private void btnDoIt_Click(object sender, EventArgs e)
{
if (_theNewForm.InvokeRequired)
{
_theNewForm.Invoke((Action)(() =>
{
_theNewForm.Text = "Hello World";
}));
}
else
{
_theNewForm.Text = "Hello World";
}
}
OK, so it’s making the Invoke call on the _theNewForm member. Now we need the this parameter of the btnDoIt_Click method, and !CLRStack –a gives it to us:
0:000> !CLRStack -a
OS Thread Id: 0x1e78 (0)
[…snipped…]
002eee44 00660595 MyNiceApp.MyMainForm.btnDoIt_Click(System.Object, System.EventArgs)
PARAMETERS:
this = 0x0261cd7c
sender = 0x0263a224
e = 0x0264de00
LOCALS:
0x002eee50 = 0x0264fda4
0x002eee5c = 0x00000000
And now let’s take a look at the _theNewForm field:
0:000> .load c:\temp\psscor2\x86\psscor2
0:000> !DumpField -field _theNewForm 0x0261cd7c
Name: System.Windows.Forms.Form
MethodTable: 708ed878
EEClass: 706acb84
Size: 320(0x140) bytes
GC Generation: 0
Fields:
[…edited for brevity…]
4 System.Object 0 instance 00000000 __identity
8 ...ponentModel.ISite 0 instance 00000000 site
c ....EventHandlerList 0 instance 02642f0c events
108 System.Object 0 static 00000000 EventDisposed
10 ...ntrolNativeWindow 0 instance 026421d8 window
[…snipped…]
0:000> !do 026421d8
Name: System.Windows.Forms.Control+ControlNativeWindow
MethodTable: 708f0e14
EEClass: 706d62b8
Size: 56(0x38) bytes
GC Generation: 0
Fields:
[…edited for brevity…]
4 System.Object 0 instance 00000000 __identity
18 System.IntPtr 1 instance 723580 handle
8 ...veMethods+WndProc 0 instance 02642f5c windowProc
1c System.IntPtr 1 instance 5180642 windowProcPtr
[…snipped…]
Now we have the window handle—723580–which we can inspect with Spy++ to find the owning thread.