Comparing TDD with ATDD

February 14, 2011

In the project I’m working on right now we’re using FitNesse as a core part of our development process and I found it very useful.

FitNesse is a wiki-like tool that provides the ability to define tests (and run them) by writing them in a special mark-up language, rather than in a programming language. This is usually best for describing user-stories (rather than low-level unit-tests), and therefore this approach is often referred to as “Acceptance Test Driven Development” or shortly ATDD.

Note: other tools (like Cucumber and SpecFlow) refer to a similar approach as Behavior Driven Development (BDD).

Our policies also guides us to work in a TDD (unit-test) approach, but at times we felt it’s redundant. Therefore initially we did TDD only on very specific class that could be easily isolated, and on classes that are not covered by our FitNesse tests, e.g. adapters with other components.

Over time (and mainly after the “englightment” I told you about in my previous post) we realized that it’s not very redundant and each approach has its own pros and cons. In fact, FitNesse User Guide summarizes the added value of each approach very nicely:

Unit tests (TDD) is about Building the Code Right
FitNesse (ATDD) is about Building the Right Code

Below I’m bringing to you a comparison of the two approaches, from several important aspects:

Focus

As I mentioned above, the focus of the two approaches is different: In TDD the focus in on testing the low-level functionality (of single classes and methods) in order to ensure good design patterns and principles (such as SRP, OCP, Cohesion, etc.), which leads to more flexible code, easier refactoring, etc..

In ATDD, the focus is on User Stories, which describe the functionality that the customer cares about, and from her point-of-view. Even though this usually means functional requirements, it’s not limited to it. Performance, scalability, and other non-functional requirements (AKA the ‘ilities’) can also be expressed as user stories and tested in this fashion.

Note that it’s definitely possible to take an ATDD approach using a unit-test framework technology (e.g. MSTest, NUnit, etc), and it’s not a bad idea whatsoever. The other way around is also technically possible, but makes much less sense (i.e. testing low-level class functionality using a tool like FitNesse)

Audience

This is maybe the key difference between the two approaches. TDD is aimed for developers, and therefore unit-tests would normally be written in a programming language. The developers are also the ones that get the most added value out of it, in the form of a flexible code that they can easily refactor and change as necessary.

ATDD is aimed mainly for the non-programmers stakeholders, first and foremost the Product Owner (or whoever defines the requirements). Tools like FitNesse can easily be used by those people to define the user stories in a precise way, and make sure that the developers implemented exactly what they meant. They also get an excellent transparency about the state and the progression of the project, as it’s simple to run all the tests and see exactly what functionality is already implemented and which still isn’t.

Ease of Adoption

Even though TDD is very beneficial once done right, the fact is that it’s not so easy to get right the first time. Even worse, when attempted but done wrong, it may cause more damage than value! TDD requires a change in the mindset of the programmers, and that’s not a trivial thing to do naturally. It takes good apprenticeship and training in order to do it right. In addition, organizations often afraid to take this approach because the managers, which are non-programmers, cannot see the great value of it, and it may look like a waste of time to write everything “twice” (once for the test and once for the production code).

ATDD on the other hand is pretty easy to adopt, and there are no major barriers that can hinder its adoption. It also takes some change in the mindset of the Product Owners, but it’s a small one (defining the requirements as testable user-stories, rather then writing abstract specs). beside of that, all stakeholders get a pretty quick value out of it:

  • Product owners and project managers get transparency and control over what was implemented. It forces them to use an exact language to describe the user stories, and inconsistencies are usually revealed early in the process
  • Developers get concise requirements and a definition-of-done. They also enjoy the safety-net that the regression suite provide when they come to do refactoring
  • The tedious work is taken away from the testers, and it frees them to do more interesting exploratory and integration testing

Maintainability

As mentioned above, good TDD is hard. If it’s done wrong (namely, tests are written sloppily) maintainability may become a nightmare. This is because now both production code and test code should be changed whenever a change in the production code is required. However, if done right, the tests assured the flexibility of the code and thus make it easier to change. If done right, you only need to change the tests that were really affected by the change in functionality and/or intention of the code (which is usually not much, and often none).

In ATDD, user-stories very rarely change. Sometimes it is required to “refactor” the tables and their corresponding fixtures to better suit the changes in the model as it evolves over time, but this is not a big deal.

ROI

The best ROI is received when combing the two approaches because of the different added value of each of them and the way the complement each other. If only ATDD is implemented, then the ROI is more apparent, but it’s also more short-term. This is because despite the safety-net that the regression suite provides, some parts of the code would eventually rot, and would be very difficult to maintain.

The ROI of using only TDD is less apparent, and takes longer to see, but after a successful adoption, the ROI of a flexible code starts to pay back in the improved maintainability over a long period of time. Note that if “only” TDD is done successfully, it implies that part of the test suite is actually acceptance tests, because good TDD is also driven by user stories.

Summary

If you consider improving the agile practices in your team by adopting a test-first approach, but you don’t have experience with TDD, I would recommend that you start off with ATDD, and gradually add TDD. This way you’ll have easier time to get everyone on-board and see the added value quickly, and go for a real TDD as a second phase.

If you want TDD and/or ATDD coaching feel free to leave me a private message, and I’ll get back to you ASAP.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*

2 comments

  1. MerlynFebruary 14, 2013 ב 7:08

    Great comparison – exactly what I was looking for. Thanks

    Reply
  2. http://www.exlair.comJuly 27, 2013 ב 7:29

    Hey that’s fantastic! Very good example here. Thanks for sharing this distinctive subject blog with us. I loved alot at the time of reading your blog. Good website.

    Reply