05 April 2009

About the usefulness of unit tests for bug fixing

I've seen this one done wrong (IMHO) so often, so I thought I just write down my recommended practice.

Using Unit tests for bugfixing
When you encouter a bug in your application you should do the following:
  1. Write a unit test which reproduces the bug
  2. fix the bug
  3. rerun the unit test to check the bug is fixed
  4. if the unit test still fails, check if either your bugfix or your unit test is flawed and go back to 2 or 3
  5. if you've changed your unit test in 4, then temporarily undo your bugfix and check that the unit test still fails iun that case
  6. deliver fix
What is the advantage of this approach?
For one this helps in fixing the bug, as you can test without any manual steps (e.g. running the whole application) that your bugfix really fixes the bug.
Additionally, the unit test serves as a regression test to prevent the bug from creeping again into later releases.

'Legacy' applications
Often you have applications which are not easily unit-testable. I can tell you a story or two about an application in which not even the tiniest parts were testable without db access (damn singletons!)
Even in that case, I try by all means to set up a unit test by refactoring small parts of the code base to make it testable. However, sometimes the bug fix is so urgent that you don't have the time to write the unit test before the bugfix. So it looks more like this:
  1. fix the bug
  2. test the fix manually
  3. deliver fix
  4. refactor code
  5. write unit test
  6. temporarily roll back the fix to test that the unit test fails
  7. repeat 4, 5 and 6, if needed
  8. deliver fix again
Why all this work (4 to 8), if the bug was already fixed?
Apart from the function as a regression test, this also helps in:
  • improving the test coverage of your legacy application which in turn enables you to do changes with greater confidence
  • improves code structure by e.g. replacing singletons with dependency injection
  • gives you more insight into the legacy code base

No comments: