March 2010 - Posts
Mark wrote a post about unit tests structure, and the need for similar tests to look the same.
Good points there. I like my tests to show their essence in a very visible way – the concrete values for the scenario and the assertion are more often than not just the primitives which vary between test case to another.
If you read the subject of this post you already know where I’m going with this: I really like row tests’ ability to concisely describe the variance between cases. So much so that I’d advocate using [RowTest] even if you have a couple of test cases (like in Mark’s example), to make the tests more readable.
As a side note – QuickUnit doesn’t support generating your tests in such a data driven manner yet, but we’ll get to it shortly.
I’ve read a couple of things that got me smiling. I am a bit tipsy, so smiling may be the manifestation of a chemical imbalance or just a nervous tick.
Karl Seguin’s post:
Ultimately, the most important thing is that you have automated tests which aren't a nightmare to setup, maintain or run. Integration tests have more dependency and thus are more fragile, but can be an efficient way to verify correctness.
Ayende’s post:
TDD fanatic corner: I don’t really care about testable code, I care about tested code. If I have a regression suite, that is just what i need.
Excuse the subject line. I wasn’t feeling creative. This post is in response to Derick’s post.
Derick asks if he should program his stubs with exact arguments per expected method call, and ends with “I know the answer is ‘it depends’”, but me - I’m a man of extremes. I don’t think it depends, and you should probably never use it (specifications), or, if you do “have to use it” – consider it a code smell for the tested code.
To me it’s like having more than one assert in you test: what happens if the method is called with “2” and not “1”? The test will fail? The stub will crash? Or is it that the real code will fail? If it is the latter – indeed you need to make sure the value is “1” and nothing else, in the scenario you test.
And you know how we do that, right?
Exactly – add another test. Add a test which checks that logic which renders that “1” argument works properly. If it really must be “1” – than we’re talking about functional requirement, and not an implementation detail, so that logic should be properly tested, and probably publicly exposed. If it doesn’t really have to be “1” there – than just don’t check it. Let the test fail on the real assertion.
And if we’re on that subject – don’t write tests with a mock object which expects a method to be “Called with exact arguments”. IT SMELLS! Separate the argument generation for the actual invocation and test them. Don’t write the simple test and disband the required refactoring – do the simple refactoring and then write the simpler test.
Yes, well… yes.
After a fascinating couple of weeks on VS2010RC – I’m going back, sir.
Simply put – performance is shite. Yes, I’m using Resharper, yes, I have a lot of XAML, and yes WiX wasn’t always very kind to me (general purpose rant), but if I actually feel that everything takes longer, and VS hangs, and takes heap loads of memory, and running my tests take longer, and I need to always think about backwards compatibility, and all the available tools are beta-ish – than I think I can manage to wait a couple of months more for the eco-system to stabilize.
Farewell, 2010RC. We hardly knew you.
(But the zoom in the edit does kick ass)
In QuickUnit we develop a Visual Studio addin which allows developers to easily create quality unit tests. It’s a rather complicated, highly extensible, code-analysis and code-generation, .net application, with a rather complex UI layer (but with a simple UX). We use WPF (on MVVM) for the UI infrastructure, and make very heavy use of viewmodel objects, data bound to data templates.
We’ve encountered a scenario where our memory consumption started growing, after every action which basically only re-rendered the UI. ProcessExplorer was kind enough to show us that the gen2 heap was growing steadily, and GC collections had no effect. Basically, the classic telltale signs of a managed memory leak.
Our first mistake was to try to find the leak manually, by code reviews. We’re a strong, two-man team, and we know the code pretty darn well, so we figured this was only a matter of catching the ill-written line or two. The second mistake was finding an ill-written line or two which made us feel good about solving the problem. J
After realizing that our solution did not help one bit – we told ourselves to get serious about it, and start profiling. Oh, the Ants Memory profiler kicks so much ass... After adding a “GC Collect” button to our application (a great feature to have in any memory profiler, btw) we were good to go.
We’ve distilled the least amount of work required to recreate the leak, took a memory snapshot for the “before”, performed that action 5 times, and then took another snapshot (actually, add “frantically pressing the GC Collect button” before any snapshot). The reason to do the action 5 times was to easily identify which objects count grew by a multiply of 5, since some objects are added for valid reasons, and do get cleaned between actions. I don’t know if it’s a best practice, but it works).
After finding such type (which has a multiple of 5 new instances) – reviewing the reference graph for one such instance is dead simple. I am, however, dead stupid, so I offset it a bit: they put a large red box at the bottom of the screen, next to the relevant instance, with a big red arrow stating “START HERE”, but I keep ignoring it, and go straight to the GC roots. Something inside me keeps telling me that it’s more important. I can’t fight it. Only after I’m persuaded to start following the graph from the right place – I can start seeing the problems.
The problem was, btw, the old familiar case of databinding to a non-INotifyPropertyChanged object, which, well… leaks. We just missed that databinding in our code review.
Other good memory leaks:
http://blogs.msdn.com/jgoldb/archive/2008/02/04/finding-memory-leaks-in-wpf-based-applications.aspx