TDD and multi-threading

March 28, 2011

no comments

According to good TDD practices a test should describe (and eventually test) a behavior and not the implementation. Because of that, one thing that is tricky to do in TDD is to write tests for a multi-threaded component.


One can argue that multi-threading is a non-functional requirement, and therefore shouldn’t be covered by behavior tests, but only by load-tests. I don’t subscribe to this approach for few reasons:



  1. As I’ll show below, it is possible to describe a multi-threaded problem as a functional requirement.

  2. Load tests are non-deterministic

  3. Load tests usually take long to run, and therefore aren’t being run very often as unit-tests

  4. The chances to break an existing multi-threaded code due to refactoring are pretty high, and so it’s very dangerous to leave it without the safety-net of unit-tests

While I claim that it’s possible to describe multi-threading requirements as functional requirements (using User Stories and Scenarios), I don’t claim that it’s easy or simple. Coming up with the right scenarios for testing a multi-threaded component require the same problem solving skills of an experienced developer if not more. Normally I wouldn’t expect from the average non-technical customer to be able to define the user scenarios by himself, and often these cases will be covered only by lower-level unit-tests and not by acceptance tests (ATDD). However, these user scenarios can be defined completely in the customer’s vocabulary, and without technical terms (like mutex, semaphore, etc.).


So let’s start with an example:

Suppose I’m developing a fax server, and for the sake of simplicity let’s assume that it can only send faxes. Users should be able to connect to this fax server and send to it a document and a fax number to dial, and the fax server sends these documents as faxes. Obviously the fax it self can handle only one document at a time.


So I can define the following user-scenario (using the given-when-then pattern):


Given: User1 started to send a fax, but still didn’t complete


When: User2 attempts to send another fax


Then: User2 should see a “Please wait” message while User1’s fax is being sent



And User2’s fax should be sent after Users1’s fax was sent completely.


This is only one scenario. You may need to describe additional scenarios in order to cover all relevant cases. But still you see, it’s possible to describe the synchronization problem as a user scenario.


Ok, but how this is “mapped” to a test in code?! How can you simulate the “given” state of “started but still didn’t complete”?! After all you don’t have a control over the context-switches that occur during the execution of your test!


If you have some experience with TDD you’re probably used to the concept of “what you can’t control – mock!”. Mocking the OS scheduler is would be the best thing, but because it do all of its work behind the scenes (no API that you can mock), then it’s not very possible to do. But you can mock the synchronization primitives (like Mutex, Events, etc). and even the Thread objects, to achieve the same goal.


Let’s see how a test code could should look like for the above scenario:



   1: public class FaxServerShould
   2: {
   3:     [Test]
   4:     [Timeout(1000)]
   5:     public void BlockAThreadIfAnotherThreadCurrentlySendsAFax()
   6:     {
   7:         UserStartsToSendAFax(_user1, _document1);
   8:         UserAttemptsToSendAFax(_user2, _document2);
   9:         AssertUserIsBlocked(_user2);
  10:  
  11:         WaitForSendingToComplete(_user1);
  12:         AssertFaxWasSent(_document1);
  13:  
  14:         WaitForFaxServerToBeIdle();
  15:         AssertFaxWasSent(_document2);
  16:     }
  17: }

Note that the TimeoutAttribute is necessary to avoid the tests block forever in case it fails due to a dead-lock. In addition, the test itself won’t contain any Sleeps so in case it succeeds it should be very fast. One thing you have to note about the TimeoutAttribute is that in case you want to debug you should comment this attribute out, otherwise NUnit will kill your thread while you’re debugging.


Clearly this test fails with compilation errors. As in TDD, that’s legitimate as we still didn’t write the code to make it pass.


You can find the complete passing test in the attachment of this post.


Notes:



  1. We could use Thread.Suspend and Thread.Resume instead of the ThreadMock class, but because these methods are depricated I created the ThreadMock class

  2. Most of the logic of the mocked synchronization is inside the LockMock class.

  3. This example demonstrates how to mock a simple lock that supports 2 threads. It can be enhanced to handle any number of threads, and similar mocks can be created for Semaphore, ReaderWriterLockSlim etc.

  4. The final production code should use a different implementation of ILock (instead of the LockMock) which will simply use the Monitor.Enter and Monitor.Exit methods.

That’s it!


I really appreciate if you leave me comments and share with me your thoughts about this post, as well as for my other posts. I try to bring up topics that I believe are both useful and interesting, and most of them are based mainly on my own ideas and experience, so I’ really love to get your feedback.

Thanks,


Arnon.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*