Ative at Work

Agile software development

Ative at Work

Why Acceptance Tests Matter

While consulting on a migration project I was refactoring the way to load country reference data when I noticed some oddities in the business rules for formatting phone and fax numbers. I had changed from loading based on hard-coded country codes to using an Enum and mapping it to the correct instance in persistence layer.

One of the methods to be refactored was a business rule saying something like:: if the country code is "DNK" (Denmark), "FRO" (Faroe Islands) or "GRL" (Greenland) then apply this kind of formatting to the telephone number.

The specification of the rule was probably reverse engineered from the legacy code.

The implementation was thoroughly unit tested and it was all green lights.

But the rule was wrong. Maybe it worked one time in the legacy system when country codes were different, but definitely not any more.

The business rule should have been specified as an acceptance test instead. 

If that had existed we would have a rule like this:

  1. Create a Danish company in the Copenhagen municipality
  2. Enter a valid telephone number such as "80 12 34 56".
  3. Verify that it is formatted correctly with no blanks, eg.. "80123456".

And we would have done the same for a company based on Greenland and one on the Faroe Islands.

Then the test would quickly have discovered that the rule was broken - as it turns out, the country code is Denmark for all Danish companies whether they are in the south, on Greenland or on the Faroe Islands. What varies is their region specifier.

So had we written the business rule specification from a business perspective and implemented an acceptance test for it we would have caught the error early. Instead, we had all green lights in the unit test but the application was broken.

While unit tests are very valuable the importance of complete end-to-end acceptance tests cannot be overstated as they are much more likely to catch errors in the specification. In fact the acceptance test suite is the best way to ensure that what you deliver not only works, but also that it does the right thing.



Søren Lindstrøm said:

I guess there is a reason why frameworks like FIT exists and should be available for developpers to run.

oktober 9, 2006 11:43

About Martin Jul

Building better software faster is Martin's mission. He is a partner in Ative, and helps development teams implement lean/agile software development and improve their craftmanship teaching hands-on development practises such as iterative design and test-first. He is known to be a hardliner on quality and likes to get things done-done.
© Ative Consulting ApS