What I Learned About Writing Unit Tests: Dependency Injection Mess With Mocks

Armed with some experience, I embraced dependency injection in all its might. I started writing subsystems and components that interact only through well-defined interfaces, which was relatively easy for me because my previous infrastructure project relied heavily on dynamically-generated proxies that worked only with interfaces. This allowed me to abstract away and stub away everything a component needed under a test.

And then my tests had the following shape and form (I’m not using any specific mock framework syntax, for illustration purposes):

[TestMethod]
public void LoggingFramework_LogToDB_Works()
{
    bool flushed = false;

    SomeMock<ILogDatabaseProvider> provider =
        SomeMockProduct.Mock<ILogDatabaseProvider>();
    provider.Expect(m => m.WriteLog).DoNothing().Once();
    provider.Expect(m => m.ReadLog).Return(new string[] { “MyMessage” });

    SomeMock<IConsoleOutput> console =
        SomeMockProduct.Mock<IConsoleOutput>();
    console.Expect(c => WriteLine).DoNothing().Once();
    console.Expect(c => c.Flush).Callback(() => flushed = true).Once();

    //…repeat for another dozen components…

    Log log = new Log(provider, console, …);
    log.Write(“MyMessage”, Severity.Critical);

    provider.Verify();
    console.Verify();
    //…all other providers—Verify()

    Assert.IsTrue(flushed, “Log console was not flushed”);
}

In the beginning, I was very impressed with the flexibility of this approach. I can over-specify the hell out of my tests, and define the subtlest behaviors for each of the methods called under test without writing a manual implementation of the mocked component tailored for each and every test.

This went very well for a couple of months, and I had no trouble at all adding more and more code and more and more tests (until I had around 50KLOC of code and 75KLOC of tests). But then, some changes in the design goals warranted a change in the system’s interfaces, and not only have the names and parameters changed, but so have the semantics. (For example, it became not OK to flush a log without messages written into it; it became not OK to write to a console unless the log was explicitly created with a console; and so on.)

I was horrified by the number of changes I had to make to my tests. Even in areas when I encapsulated some of the mocking logic to a separate function, I had to rewrite more test code lines—by an order of magnitude—than the number of lines I changed in the actual code.

Apparently, this is a well-known phenomenon of overly-specified and thus very brittle tests. I had to learn it the hard way. In the next installment, I should hopefully wrap up this series by explaining the more pragmatic approach I now use when writing unit tests.