Lancet Then Lance It

Short arms and deep pockets, that’s me. Not that I’m a total skinflint (although I’ll admit to thrifty) but I do like to try to get good value out everything that I do. Sometimes this can be planned and sometimes I get it by exploiting existing resource. With test automation, one way to get value is to write code that can do double duty – as a surgical tool, crafting data and scenarios for those very specific checks, and by simply multiplying it up, as a long stick to whack the application with.

For example, I might think about writing test code to run these ways, as a second-order priority:

• parallel: Look to run multiple instances of the tests in parallel without them interfering with one another. Write data to unique locations per instance of the test to ensure this. Check that any data that the suites are dependent on is not altered in the execution of the suite.
• serial: Look to run the same suite multiple times against the same application in series, perhaps in a tight loop, without each run interfering with the previous. Consider carefully whether each instance of the test code leaves the application in a state where is should be conducive to another run of the test. Check particularly for cruft left behind by tests, either in the application’s file space, database, memory and so on, or in your test output so that you don’t misinterpret the results of the tests.

In search of even greater value, it can be interesting just to run existing tests in parallel without even considering whether they’re suitable. While you’ll probably encounter test failures due to interference, you might well find gross errors in the application due to load, or bad validation of the bad test data, broken requests and so on. Likewise, leaving some tests running in an infinite loop can help to provoke memory or disk space issues. If you go down these routes, as complete logging as possible will help post hoc diagnosis of the issues that you find.
Image: http://flic.kr/p/9agAJm