We had to update a mature regression test suite recently. The task was effectively moving a set of tests from a source data set that was no longer supported to its replacement. For product reasons this wasn’t a trivial undertaking, but we could anticipate most of the problems and decide in advance on a strategy that would make the migration as straightforward as possible.

Unfortunately, as the source had been in the test system for several years dependencies had built up around it. Dependencies that were reasonable pragmatic choices at the time, or forced short-cuts to meet a deadline or just plain wrong but dependencies nonetheless. Of course, as soon as we retired the old source data, a selection of apparently unrelated tests started falling over.

Because this is the real world, our remedial action has only removed the dependencies in some cases and just made the suites run again in others. We’re filing bug tickets for ourselves (you do maintain a bug database for your tests don’t you?) against the latter cases and we’ll hopefully have opportunity to really fix them before we go through this again.

This is a pain for us, but think about your colleagues in Development for a minute. Their codebase is probably older and more mature than yours, much larger and more diverse than yours, gets hacked about by more hands than yours and changes much more frequently than yours ever will. Even with careful handling and refactoring, unwanted dependencies are introduced. Every little bug you report has the potential to bring something seemingly unrelated crashing down around their ears, and they know it.

In this area, as in so many others in testing, me and Mick Jagger see things the same way: we’ve got sympathy for the Dev team.