December 2011 - Posts
A few days ago I delivered a session on return-oriented programming, in the context of stack-based buffer overflow exploitation, at the Distributed Systems, Networking and Security seminar (HUJI).
Generally speaking, return-oriented programming (at least in limited form, such as return to libc, return to syscall) is not new at all. It is a very effective means of bypassing stack-based buffer overflow mitigations such as NX (non-executable stack) and W+X. The awesome thing about ROP is that code execution vulnerabilities don’t have to involve actual code being placed in memory – a carefully constructed sequence of stack words can lead to arbitrary code execution through pieces of code (ROP gadgets) located elsewhere in memory.
However, my presentation was based mostly on results from a 2011 paper by Shacham et al., where they show that Linux and Solaris libc binaries contain more than enough ROP gadgets to enable arbitrary control flow, and develop an actual compiler for generating exploit stack structure from a C-like syntax. The paper is short, funny, and highly recommended.
If you’d like to read my short presentation instead, view my slides here.
Last Wednesday I delivered my last session at the SDP: Production Debugging of .NET Applications. After delivering a similar session in the June DevDays, I thought about how I can make it better by focusing on a smaller set of core debugging scenarios and making sure attendees get a chance to practice them first-hand.
Indeed, we had time to discuss and practice the following:
- Capturing crash dumps and hang dumps with ADPlus, Windows Task Manager, and Procdump
- Analyzing crash dumps in Visual Studio 2010 and WinDbg to find the exception that occurred and the stack of other threads
- Detecting memory leaks while the application is running or from an out-of-memory postmortem crash dump using ANTS Memory Profiler and .NET Memory Profiler
- Automatically looking for deadlocks using SOSEX’s !dlk command
If you’re looking for references and sample scenarios, make sure to check out my blog post: .NET and C++ Debugging Resources. I use it every time I teach my full .NET Debugging Workshop, which is packed with >20 hands-on debugging labs.
If you attended my session, thanks a lot for coming and I hope you’ve seen a glimpse of what production debugging can look like. The intimidating task of opening crash dumps or analyzing complex bugs can be fun if you have a set of core scenarios and patterns, and a great toolbox.
I have been recently posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
Noam and I delivered a joint keynote at the first day of the SDP, covering the new APIs and internal features of .NET 4.5. With my love for internals, I took the easy route of talking about CLR internals and C# 5 async methods, and Noam talked about WCF, Entity Framework, WPF, ASP.NET MVC, and plenty of other frameworks which have been updated in .NET 4.5.
As you probably know, .NET 4.5 is an in-place update for .NET 4.0, which means—as far as Microsoft is concerned—that it should work seamlessly where .NET 4.0 does, with full backwards compatibility. Indeed, I’ve taken the plunge and installed Visual Studio 11 Developer Preview (which ships with .NET 4.5) on my primary laptop and desktop, and haven’t run into any trouble during the last >2 months.
I spent a large part of my 20 minutes talking about C# 5 async methods, which are definitely on their way to becoming my favorite C# feature. It’s going to be very hard to imagine how we used to work directly with continuations and marshaling work back to the UI thread after the dust settles and Visual Studio 11 is released. I haven’t had time to show much more than the simple examples, but I’m pretty sure the potential was clear:
async void DownloadLargeFile(string url) {
_cts = new CancellationTokenSource();
try {
byte[] result = await DownloadAsync(url, cts.Token);
PlayMovie(result);
} catch (OperationCanceledException) {
UpdateStatus("Download canceled.");
}
}
void CancelDownload() { _cts.Cancel(); }
(Async method with cancellation support and exception handling, all covered by the C# 5 compiler. To run this sample, you will need the Visual Studio 11 Developer Preview.)
During the last few minutes of the talk I focused on the latest CLR performance improvements: concurrent background server GC, multi-core background JIT, automatic NGEN, and managed profile-guided optimization. Currently there aren’t many resources on this topics lying around, but the //build session is pretty good and detailed.
It’s the first CLR release in >5 years that brings serious news from the performance front, especially w.r.t. compilation time penalty. I am looking forward to testing the final bits on production systems to see how startup time and CPU utilization are affected by these impressive features.
I have been recently posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
The SDP started with my 40-minute keynote, Introducing Windows 8. I was working on it for more than 3 weeks, and wasn’t completely sure what I wanted in it until only a few days before the conference. That was also when I decided to ditch the slides and go for a fresh idea: a Metro-style Windows 8 application that contains both the slides and interactive code demos for the session.
(The application’s tile and title page.)
My personal view of Windows 8, after letting the news sink and playing with the system for a couple of months, can be summarized in the following three tenets:
Building on the foundation of Windows 7—the new OS has plenty of new features, but it doesn’t shake the foundations and maintains full compatibility with existing Windows 7 applications, albeit in the desktop environment.
Fresh experience for new form factors—the new Metro touch-centric UI targets the tablet form factor as well as hybrid laptops with touch screens. On these form factors, Metro is a revolution.
New application model and development framework—it’s impossible to achieve good battery life, provide a fresh start for application development, and deliver a brand new UI experience without changing the way applications interact with Windows and with each other, and the fundamental API sets that they can use.
Most of my session focused on the various scenarios, features, and APIs that Windows 8 provides to Metro applications. I mentioned language projections to the various languages—C#, VB, C++, JavaScript—and how the WinRT APIs map naturally to the C# 5 “await” concept or to JavaScript promises.
Below are some of the code demos I’ve shown:
Updating live tiles from application code (as opposed to push notifications)—I created a badge on the application’s tile and offered the user to pin a secondary tile to the start screen. This secondary tile would deep-link into the application, to a specific screen.
Uri logo = new Uri("ms-resource:images/Logo.png");
SecondaryTile secondaryTile = new SecondaryTile(
"SDPKeynoteSecondaryTile",
"SDP Keynote Tile",
"SDP Keynote - Tiles Deep Link",
"SecondaryTile",
TileDisplayAttributes.ShowName,
logo);
bool isPinned = await secondaryTile.RequestCreateAsync();
(Pinning a secondary tile to the start screen, pending the user’s permission.)
Next, I showed how pickers can be used to integrate with other applications and with Windows without implementing specific interfaces and being aware of all data providers on the system. Namely, I used the image picker to choose one of my Facebook photos (through the Socialite sample application) and use it in my app.
FileOpenPicker picker = new FileOpenPicker();
picker.FileTypeFilter.Add(".png");
picker.FileTypeFilter.Add(".jpg");
picker.SuggestedStartLocation =
PickerLocationId.PicturesLibrary;
picker.ViewMode = PickerViewMode.Thumbnail;
var result = await picker.PickSingleFileAsync();
var stream = await result.OpenAsync(
Windows.Storage.FileAccessMode.Read);
BitmapImage image = new BitmapImage();
image.SetSource(stream);
ResultImage.Source = image;
(Using the FileOpenPicker to ask Windows for a .jpg image from any data source the user will choose.)
Next, I talked about the sandbox model for Windows 8 Metro applications, how each application runs under its own user identity and cannot access the resources of other applications, and how access to privacy-affecting data (such as location or camera) requires the user’s explicit permission after declaring this intent in your application’s manifest.
CameraCaptureUI capture = new CameraCaptureUI();
var result = await capture.CaptureFileAsync(
CameraCaptureUIMode.Photo);
var stream = await result.OpenAsync(
Windows.Storage.FileAccessMode.Read);
(Using the CameraCaptureUI to capture a picture from the device’s camera.)
Next, I showed how contracts bridge together applications that are blissfully unaware of each other, and enable unforeseen sharing scenarios that will feel natural to every Windows 8 user.
Finally, I talked about LiveID and roaming settings, which is an incredibly easy way to set up your Windows 8 machine, but also an incredibly easy way for application developers to share application settings and data (game levels, achievements, favorites, history, last read book page, …) across devices. With Windows 8 on the desktop, laptop, and tablet, users will appreciate roaming-savvy applications that keep up with the multitude of devices.
I didn’t have much time to answer questions, although I did prepare a slide with FAQs. Later during the day, many people asked me if Silverlight is dead, what the porting process from other UI frameworks looks like, how WinRT is implemented, and what restrictions apply to Metro applications running in the background. Answers to all these questions—and many others—were provided by Tomer and Elad in their back-to-back Windows 8 full-day sessions, later in the SDP.
To paraphrase Steve Ballmer, these are exciting days for Windows developers, and an exciting time to become a Windows developer. I’m sure there will be many more Windows 8 news to come. In the meantime, thanks for coming to the SDP—or for reading this post—and make sure to share your Windows 8 experiences, too!
I have been recently posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
Noam and I delivered on Tuesday a joint session called Everything New in C++ at the SELA Developer Practice. It’s been a really fun session to work on, even though it was also a cold reminder how easy it is to forget “The C++ Way” when you stay away for a little while. The new C++ standard is not just a set of minor additions to the C++ language and libraries—it almost feels like a whole new language, what with the lambda functions, type inference, and rich concurrency libraries.
While we were planning this full-day C++ session, Noam and I decided to focus not only on the standard C++, but also on some of the emerging extensions and runtimes around it. Because this day was the only day aimed directly at C++ developers, we wanted to cover as much of the new ground as possible. And indeed, we did a lot:
First, I talked about the new C++ standard. We couldn’t discuss in depth all the features, but obviously the first to mention were automatic type inference (a.k.a. auto), lambda functions, and rvalue references. Explaining the motivation for rvalue references—especially around perfect forwarding—is always tricky, and it’s been great to see an understanding audience of seasoned C++ developers applaud the reasoning behind the new language features.
int fib1 = 1, fib2 = 1;
auto next_step = [&]() {
int temp = fib2; fib2 = fib2 + fib1; fib1 = temp;
};
for (int i = 0; i < 20; ++i) next_step();
cout << fib1 << " " << fib2 << endl;
int n = 10;
auto say_yes_n_times = [=]() mutable ->bool {
return (--n > 0);
};
(Two examples of lambda functions, the biggest feature—in my opinion—in the C++11 standard.)
Next, I talked about the “new STL”—the additions to the C++ standard library that made it through the standard, with a specific focus on concurrency (std::thread and its kin). I also mentioned regex support and unordered containers, which should have been part of the STL a long time ago.
template <typename Future>
void wait_all(initializer_list<Future> l) {
for each (const auto& f in l) { f.wait(); }
}
template <typename RAIter>
void quick_sort(RAIter begin, RAIter end) {
RAIter p = partition(begin, end);
auto left = std::async([=]() { quick_sort(begin, p); });
auto right = std::async([=]() { quick_sort(p++, end); });
wait_all({left,right});
}
(Naïve parallel QuickSort using lambda functions, the async library function, futures, and initializer lists.)
Next, Noam talked about ConcRT—the Microsoft-specific concurrency runtime for C++ applications which shifts concurrency from being about threads to being about tasks and data-parallel algorithms. Noam used a couple of great textbook examples taken from the Win32 Concurrent Programming course that demonstrate exactly how easy it is to break recursive algorithms into tasks and to parallelize data-oriented algorithms with the data-parallel APIs.
combinable<int> sum;
parallel_for_each(matrix.begin(), matrix.end(), [&](row& r)
{
for each (int i in row) sum.local() += i;
});
int total_sum = sum.combine(
[](int a, int b) { return a+b; });
(Using the parallel_for_each ConcRT algorithm to parallelize matrix summation with partial sums for each thread aggregated into a combinable<int>.)
Finally, we moved to unexplored-land, the emerging-but-not-implemented-and-completely-nonstandard extensions to C++. Noam talked about C++/CX (a.k.a. /Zw), the language extensions for working with WinRT in Windows 8. These neat language extensions (very similar to C++/CLI) make consuming and exporting WinRT components very easy despite their COM-laden nature.
I had very little time left to talk about C++ AMP, a set of APIs and minor language extensions which make standard C++ code run on the GPU with as little extra work as a lambda function:
void MatrixMultiplyAMP(
vector<float>& vC, const vector<float>& vA,
const vector<float>& vB, int M, int N, int W) {
array_view<const float,2> a(M,W,vA), b(W,N,vB);
array_view<writeonly<float>,2> c(M,N,vC);
parallel_for_each(c.grid,
[=](index<2> idx) restrict(direct3d) {
int row = idx[0]; int col = idx[1]; float sum = 0.0f;
for(int i = 0; i < W; i++)
sum += a(row, i) * b(i, col);
c[idx] = sum;
});
}
(Matrix multiplication using C++ AMP, non-tiled, adapted from Daniel Moth’s blog post.)
Thanks for attending this session, and we truly hope that you like the direction C++ is taking. It’s really the renaissance of C++ in the 2010s, starting with Visual Studio 2010 and the new C++ standard, and going forward with Windows 8 support, general purpose GPU programming, and game development. You can download some of the code demos and exercises we used during the day from here.
I have been recently posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn
I apologize for the silence during the last two weeks—organizing the SDP and preparing three full-day sessions and two keynotes left no time to breathe :-)
On Monday I delivered a session called Improving the Performance of .NET Applications at the SELA Developer Practice. Here are some of the practical scenarios we covered:
- Measuring application memory usage and allocation sources
- Diagnosing memory leak sources with memory profilers
- Using sampling and instrumentation profilers to find CPU bottlenecks and methods with problematic cache access patterns
- Reading performance counter information as a lead into more intensive diagnostics
Additionally, we’ve had a couple of hours to talk about more “theoretical” things such as the memory layout of .NET reference types, boxing and its true implications, workstation and server GC flavors, and GC generations.
It’s been the second time I’m doing this session (the first was at the DevDays in June), and the room was packed again. I thought then, as I do now, that learning how to measure application performance is impossible without some practice time with the tools – so this time we created a unique format with 7 hours of frontal training followed by 2 hours of self-paced hands-on exercises in a computer classroom. I think this workshop format is the best way to experience performance measurement and optimization in a single-day setting.
If you participated in this day, thanks a lot for coming, for your constructive comments and for your interesting questions. If not – you can always take the full course, or else I’ll see you at the next SDP in May :-)
Other sessions delivered today included Windows Phone Mango (by Alex Golesh), HTML 5 (by Gil Fink), WCF Crash Course (by Erez Harari), and Windows Azure (by Manu Cohen-Yashar). Tomorrow Noam and I are up to talk about the new C++ standard and other C++ goodies—stay tuned!
I have been recently posting short updates and links on Twitter as well as on this blog. You can follow me: @goldshtn