The concept of test doubles is well-established in unit testing with mocking probably the most familiar. The idea is that you fake enough of an API to permit unit tests to run against it. You can control the way that the mock API responds, tailor your test coverage and avoid external executables, dependencies on other code and so on to run your tests.
It can be useful to use similar concepts at the component level too. For example:
- for diagnosis and investigation of misbehaviour in complex systems you can replace components in a workflow. I like this especially when the problem behaviour is hard to reproduce naturally. Part of our core product is a server that calls out to other software for some tasks and returns data back to a client. Replacing a server-side executable with something that generates a specific output (e.g. returning an error response to the server, or bad data to the client) or behaviour (e.g. taking so long to respond that the server is forced to time out) can be extremely helpful in identifying a cause or prompting an effect.
- similarly, you can capture input data for a component by replacing it with a script that simply dumps its arguments to some convenient location. This is particularly useful when you have no doc and/or the product logging doesn’t give you enough information, or you don’t trust it.
- in regression tests it can be useful to isolate individual executables and having them talk to a shim rather than another real component. I’d almost always also run the real components in tandem too, for the potential interaction side-effects. Once you have the framework, multiple variant runs can be cheap.