What I Learned About Writing Unit Tests: Dependency Injection Mess With Mocks

October 18, 2009

4 comments

Armed with some experience, I embraced dependency injection in all its might. I started writing subsystems and components that interact only through well-defined interfaces, which was relatively easy for me because my previous infrastructure project relied heavily on dynamically-generated proxies that worked only with interfaces. This allowed me to abstract away and stub away everything a component needed under a test.

And then my tests had the following shape and form (I’m not using any specific mock framework syntax, for illustration purposes):

[TestMethod]
public void LoggingFramework_LogToDB_Works()
{
    bool flushed = false;

    SomeMock<ILogDatabaseProvider> provider =
        SomeMockProduct.Mock<ILogDatabaseProvider>();
    provider.Expect(m => m.WriteLog).DoNothing().Once();
    provider.Expect(m => m.ReadLog).Return(new string[] { “MyMessage” });

    SomeMock<IConsoleOutput> console =
        SomeMockProduct.Mock<IConsoleOutput>();
    console.Expect(c => WriteLine).DoNothing().Once();
    console.Expect(c => c.Flush).Callback(() => flushed = true).Once();

    //…repeat for another dozen components…

    Log log = new Log(provider, console, …);
    log.Write(“MyMessage”, Severity.Critical);

    provider.Verify();
    console.Verify();
    //…all other providers—Verify()

    Assert.IsTrue(flushed, “Log console was not flushed”);
}

In the beginning, I was very impressed with the flexibility of this approach. I can over-specify the hell out of my tests, and define the subtlest behaviors for each of the methods called under test without writing a manual implementation of the mocked component tailored for each and every test.

This went very well for a couple of months, and I had no trouble at all adding more and more code and more and more tests (until I had around 50KLOC of code and 75KLOC of tests). But then, some changes in the design goals warranted a change in the system’s interfaces, and not only have the names and parameters changed, but so have the semantics. (For example, it became not OK to flush a log without messages written into it; it became not OK to write to a console unless the log was explicitly created with a console; and so on.)

I was horrified by the number of changes I had to make to my tests. Even in areas when I encapsulated some of the mocking logic to a separate function, I had to rewrite more test code lines—by an order of magnitude—than the number of lines I changed in the actual code.

Apparently, this is a well-known phenomenon of overly-specified and thus very brittle tests. I had to learn it the hard way. In the next installment, I should hopefully wrap up this series by explaining the more pragmatic approach I now use when writing unit tests.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*

4 comments

  1. RobertNovember 8, 2009 ב 10:19 AM

    Eagerly awaiting the next chapter in this series 🙂

    Reply
  2. Niall ConnaughtonJanuary 20, 2010 ב 12:45 PM

    Any chance of that final instalment? 🙂

    Reply
  3. VitaliyFebruary 25, 2011 ב 12:58 AM

    The unit tests should reflect the specification of the unit. You must mock everything that is part of its public behavior, not related to implementation details. Which means that if the semantics (=spec) changes, as in the case you are describing, it is perfectly O.K for the unit tests to change. It is quite impossible to write tests that do not change with the behavior of the unit because that is exactly what you are testing.

    In the example above, It seems you are making two test in one unit test, namely the db and the console.

    To make a less brittle test suite, you probably should test all the providers independently. This way you can change the behavior of a certain provider w/o invalidating the tests for all the other components.

    I must admit that Writing good, stable and readable tests is one of the hardest challenges I encounter as a developer.

    Reply