DBX vs. Visual Studio and WinDbg: Part 1, Calling Functions

April 5, 2011

no comments

I’ve recently had an enlightening experience teaching a C++ Debugging course to a group of developers who are transitioning from a Solaris environment to Windows and Visual Studio. This hasn’t been an easy transition for them, and the course wasn’t easy to teach—specifically, because one of the most discussed topics was feature parity between DBX (the debugger of choice for C++ applications on Solaris) and Visual Studio. Fortunately, the course focuses on WinDbg, which has alternatives to several debugging features that are inaccessible from Visual Studio; and enabled me to address at least partially the pain points and missing features after leaving DBX.

Following the course, I decided to write a series of posts outlining the unique features of DBX and how they can be emulated using Visual Studio and WinDbg. The purpose is not to convert us all to loyal DBX users, but rather to see how some features we may never have considered are taken for granted on other platforms.

Some ideas I have for this blog series:

  • Calling a function [this post]
  • Configuring breakpoints
  • Tracing execution
  • Execution control
  • Displaying data, including STL collections
  • Runtime application checking
  • Miscellaneous commands

In this post we’ll discuss a fairly useful feature—calling a function in the middle of the debugging session. The function may belong to the current execution path, or any other library that is currently loaded into the process.

DBX makes calling a function from the middle of the debugging session rather easy with the “call” command, and supports virtual methods, static functions, and arbitrary parameters. You can even set a breakpoint in a function you call that way, and stop to examine the program’s state. For example:

(dbx) list
   11     std::map<int,std::vector<float> > m;
   12     std::vector<float> v;
   13     v.push_back(4.0f);
   14     v.push_back(5.0f);
   15     m[5] = v;
   16     global = 2;
   17     getchar();
   18   }

(dbx) next
stopped in main at line 16 in file "stl.cc"
   16     global = 2;
(dbx) stop inmember size
(3) stop inmember size
(dbx) print m[5].size()
stopped in std::vector<float, std::allocator, <float>void>::size at line 375 in file "stl_vector.h"
  375   size() const { return size_type(end() – begin()); }
dbx: Stopped within call to ‘size’. Starting new command interpreter
(dbx) where
=>[1] std::vector<float, std::allocator, <float>void>::size(this = 0x8067a54), line 375 in "stl_vector.h"
  ———- called from debugger ———-
  [2] main(), line 16 in "stl.cc"
(dbx) pop -c
dbx: Call to ‘size’ aborted. Going back to previous command interpreter

Visual Studio
The Visual Studio equivalent of this feature is, of course, the Immediate Window. Unfortunately, the limits of what does and what doesn’t work in the Immediate Window are rather vague, and experimentation is in order. First of all, you can call virtual methods, static methods, or whatever else you want on local variables, memory addresses, etc., and pass to them parameters. There are some limitations e.g. on overloaded operators, but generally you can live with that. Here are some things you can expect to work:

v.capacity() – v.size()

[1]((5, [2](4.0000000,5.0000000)))
    [comp]: less
    [0]: (5, [2](4.0000000,5.0000000))

((std::vector<float,std::allocator<float> >*)(0x0046fb28))->reserve(50)

To call a function from another library, the strange {,,} context specification syntax needs to be used:

{,,msvcr100d}printf("Hello World!\n")

Neither I nor other bloggers I found on the web have been able to decipher the context specification syntax fully, other than the fact that the last part is used to specify the name of a DLL. The official documentation on this seems simply wrong.

Mixing and matching the context specification evaluator and local variable names does not work; neither does mixing a standard function call (like v.size() above) in an expression that uses a context specification.

Also contrary to the documentation, breakpoints that you set (or even DebugBreak calls) in the invoked function are not fired, so you cannot effectively inspect the executed code. An alternative would be moving the instruction pointer to the function you want to execute using the Set Next Statement command, and then retracting the instruction pointer—this is very fragile, but may work under certain circumstances.

All in all, the Immediate Window support for function invocation is rather half-baked.

On the surface, WinDbg has no support at all for calling arbitrary functions from the middle of the debugging session. Fortunately, there is a debugging extension called SDbgExt which has been very useful to me in the past. This extension makes available commands for loading DLLs dynamically and calling methods on them from the debugger session, namely the !remotecall command.

Calling a global function is rather easy with SDbgExt, but calling a member function on a local variable—something even the Immediate Window excels at—is not trivial, because you need to pass the this parameter to the method manually. However, breakpoints in called functions work seamlessly, because what SDbgExt does is create a thread that calls the function and returns a result. For example:

0:001> bm "myapp!std::vector<float,std::allocator<float> >::size"
  1: 01292900 @!"myapp!std::vector<float,std::allocator<float> >::size"

0:001> !remotecall
Usage: !remotecall <address> <call-conv> [arguments]
Calling conventions are specified as integral values: stdcall(0), cdecl(1), fastcall(2)

0:000> dv /t /v
0030f7d4 int argc = 1
0030f7d8 wchar_t ** argv = 0x001e84c0

0030f780 class std::vector<float,std::allocator<float> > v …

0:000> bl
1 e 01292900 [c:\program files (x86)\microsoft visual studio 10.0\vc\include\vector @ 878]    0001 (0001)  0:**** myapp!std::vector<float,std::allocator<float> >::size

0:000> !remotecall 01292900 0 0030f780
myapp!std::vector<float,std::allocator<float> >::size() will be run when execution is resumed

0:000> g

myapp!std::vector<float,std::allocator<float> >::size() [conv=0 argc=4 argv=00030488]
Breakpoint 1 hit

myapp!std::vector<float,std::allocator<float> >::size:
01292900 55              push    ebp

0:002> k
ChildEBP RetAddr 
00b6f974 00f01052 myapp!std::vector<float,std::allocator<float> >::size [c:\program files (x86)\microsoft visual studio 10.0\vc\include\vector @ 878]
WARNING: Stack unwind information not available. Following frames may be wrong.
00b6f9f0 0003046a sdbgext+0x1052
00b6f9f4 00030000 0x3046a
00b6f9f8 00000000 0x30000

Although it isn’t perfect, and definitely not as convenient and DBX’s call command, !remotecall is a useful and predictable alternative.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>