Ative at Work

Agile software development

Ative at Work

The TDD Controversy - JAOO 2007

TDD will ruin your architecture! When Jim Coplien said this in his presentation about Scrum and Architecture at the JAOO 2007 conference he really got people's attention. His statement sparked one of the most passionate debates at the conference.

On Roy Osherove's initiative we did an open-space session discussing the topic.

The controversy seemed to be related a lot to what TDD really means. It became quite clear that TDD means different things to different people.

Starting with testing we did reach consensus on some key issues, however:

  • Automated testing is a key enabler for agile development.
  • Test-first is the most effective approach for this.
  • Test-first on the micro-level such as writing a "SaveModifiedInstanceShouldUpdateAuditTrail" test for a Save method seemed to be acknowledged as a Good Thing. Micro-level here means developer testing on a sufficiently low level of integration. There was no consensus on whether to call this unit testing or subsystem/component testing.
  • Testing only the return values of methods has limited benefit, and many of the people attending did not write tests for simple state like property getters and setters. Also, Jim mentioned some studies showing that only a small percentage of actual bugs are related to state.
  • Testing the behaviour - especially the interactions between objects - is where the real value is at. This is one of the reasons that mocking frameworks are so popular.

There was no clear consensus on whether or not to use a test-first approach for testing the application from a business perspective, ie. testing it on the macro level. One argument was that you need to have a little bit of the application in place before it makes sense to integrate it so it would not be truly test-first. However, no-one questioned the value of automated testing on the macro-level (see the post on why acceptance tests matter) and we all recommend doing it at the earliest possible moment.

The microlevel testing mentioned above is what I usually call unit testing (although it may not be what unit testing originally meant). It involves a class under test in a controlled environment. This means that we have mocked the interesting classes that it interacts with, but full domain classes may be used as well. For example, if we are testing a mortgage calculator that takes a house and a loan profile as inputs and interacts with pricing service I would usually recommond creating a semantically valid, complete house with the real domain classes (from a factory - the ObjectMother pattern). I would explicitly create the loan profile (amount, duration, interest rate etc.) in the test since it contains the "interesting" variables related to the calculation and I would mock out the service that provides the current ticker price for loans by interest rate that the service uses to calculate the mortgage. The point is that even though I call it a unit test we are at a low but non-trivial level of integration.

Learning to be good at this takes some time since it is easy to write bad tests - in fact, Roy is currently writing a book on this called The Art of Unit Testing to help beginners climb this curve. Jim mentioned Behaviour Driven Development several times as an example of how to do this. From my personal perspective BDD looks like an elegant expression of where you can go with micro-level testing and experience.

Now, what about TDD then? Jim quoted some studies of student programmers showing that TDD done strictly from the YAGNI principle leads to an architectural meltdown around iteration three. One of the examples he mentioned was that of a bank account where TDD would lead to modelling the account as the balance (possibly with an audit trail) whereas a domain analysis would lead to modelling it as a collection of transactions with the balance being a function of these transactions.

However, let's also note that it is entirely possible to do bad architecture without TDD. From my perspective it is also very hard to do no architecture even with TDD if you have experienced people on the team. The reason is that they will know sound patterns and structural concepts that they will use, even unconsciously. For example, Jimmy Nilsson wrote a book to define a default architecture for .NET business applications, and Ruby on Rails provides a common architecture for web applications. Using these appropriately help keep your large-scale structures sound. This is one of the reasons why alpha architects working in known territory can avoid architectural meltdown in the large even with TDD. You can still make mistakes in the domain-specific modelleing. In fact I don't recall a single sizable project, big upfront design or not, where we did not learn something that would guide us to a better architecture if we had to do it again.

But, alas, as someone said in the conference: Good judgement comes from experience - and exerience comes from bad judgement. The goal is not necessarily to avoid failure entirely but rather to make it short-lived and inexpensive to correct. We are aiming for cost-efficiency rather than paralysis.

I will leave the controversy at this. As you can see there is a consensus about most of the issues related to test automation and test first. If that's your definition of TDD then by all means, please keep doing it!

Published sep 28 2007, 09:12 by Martin Jul
Filed under: , ,



Tennessee Leeuwenburg said:

I am currently planning a medium-large project in my head, and am trying to scope out all the things which I will need to get nailed down before setting forth on the good ship "entrepreneur".

Managing development will be one of them. I've had the (mis)fortune to mostly work on existing software, meaning I have just had to fit in with whatever was already in place (usually not much). If you were starting a new project from the ground up, what would you think of:

 * TDD in helping manage communication between developers?

 * TDD during prototyping and rapid development

 * TDD and design specification?

 * TDD from the client perspective?

oktober 5, 2007 6:37

Michael Foord said:

"TDD done strictly from the YAGNI principle leads to an architectural meltdown around iteration three"

But with short iterations refactoring your model at iteration three is not only easy but expected. :-)

It doesn't violate YAGNI (in my opinion) to have some basic understanding of your problem domain and an awareness of architecture issues.

oktober 5, 2007 1:05

Martin Jul said:


Let me try to answer your questions without using the "TDD" word.

* TDD in helping manage communication between developers?

In my experience a good set of automated tests written well is an excellent vehicle for communicating the intent of the programme. Ths means that they serve as a communication mechanism that can replace a lot of documentation.

On one very large project where we had a set of around 20 rules that all domain classes had to adhere to

I created some "cross cutting tests" that would reflect through the .NET assemblies and test these rules against all the domain classes. Overall it was a RUP project with a lot of documentation. This simple testing practise caught many more errors of compliance to the rules than the code and documentation reviews - and since we could run it with every build it took us no extra effort. Also, it was a trigger to talk to people about how to write good code so they would learn to avoid making the same mistake over and over.

However, I also think it is a good idea to have a short developer handbook with the project with pictures of the central concepts to give a quick visual introduction to the key architectural elements. In my experience pictures of behaviour (sequence diagrams etc) are muh more useful than pictures of the static structure such as class diagrams (these can easily be generated from code anyway).

See my article:

* TDD during prototyping and rapid development

Normally I always advice test-first - also during prototyping and especially if you want to do rapid development. The fact is it takes less time to prevent errors from creeping in than putting them in, seeking them out and removing them again. So I would definitely go for writing tests first.

* TDD and design specification?

You definitely need to have an architecture. Jim Coplien adviced to spend a minimum of three days of architecture up-front (and a maximum of one month). Usually if we work in known territory we will do this very quickly since we have an experienced team we can say, "let's go with the overall architecture from project XXX with these modifications..." That will get is into a known good zone quickly and then we can focus our effort on the application domain-specific stuff, not so much the architectural infrastructure things like organizing into layers, which application container and persistence technology to use etc.

* TDD from the client perspective?

One of the findings with automated testing is that it improves the quality of the application dramatically and speeds up the development process which makes for a happy customer.

oktober 7, 2007 11:05

Martin Jul said:

Jim Coplien has posted a new article about this discussion:

Religion's Newfound Restraint on Progress

oktober 9, 2007 8:39

Martin Jul said:

Roy Osherove has also written an article about the discussion:

The Various Meanings of TDD

oktober 11, 2007 2:46

Jesper Rugård said:

Just some notes on your very sensible post.

On tests: You need to edit the number of tests that are expressing the requirements in order to make it comprehensible for other people. Often you will have to or more levels of tests: Those that show the requirements and tests these, and those that test strange corner behaviour. It can be quite confusing to navigate tons of tests in order to understand the purpose of the code (example/intentional code can sometimes help here)

On diagramming:

Auto-creation of diagrams for classes is one of the worst ideas ever, and in my experience creating a usefull class diagram is more akin to stating the metaphor of the system than to pressing the "create class diagram" button. A lot of classes used are just for support and for intermediary use as adapters etc.

On domain knowledge:

The idea of coding without trying to understand the domain is doomed (I think), you should at least use time on getting to know the context you are working in, and that takes soem time. For a new project with a new crew, I would use at least a week up front trying to get a grasp on the fundamentals - a fool with a tool and all..

november 22, 2007 9:56

magento themes said:

Managing development will be one of them. I've had the (mis)fortune to mostly work on existing software, meaning I have just had to fit in with whatever was already in place (usually not much).

april 7, 2011 11:08

Test-Driven Development Misconceptions | Agile Pain Relief said:

Pingback from  Test-Driven Development Misconceptions | Agile Pain Relief

juni 21, 2015 7:05

Test-Driven Development Misconceptions | Agile Pain Relief said:

Pingback from  Test-Driven Development Misconceptions | Agile Pain Relief

august 13, 2015 9:41

About Martin Jul

Building better software faster is Martin's mission. He is a partner in Ative, and helps development teams implement lean/agile software development and improve their craftmanship teaching hands-on development practises such as iterative design and test-first. He is known to be a hardliner on quality and likes to get things done-done.
© Ative Consulting ApS