NPR had an interesting piece on this morning about a study of inattentional blindness on radiologists. link

I was a little sad that they didn’t include ‘tester’ of any kind in their short list of professional searchers. It hit me while listening to the story how easy it is to train someone to NOT see potentially important information. Give them too detailed of instructions and most people will do exactly what yo stated, nothing more. Tell them too specifically what they are looking for, and many will miss everything else.

How does this relate to testing? I see it as a divide between two (for purposes of this example, extreme) camps: exploratory testing vs. testing via test plans.

Personally, I like exploratory testing, I find it more freeing, and it allows me to just play with stuff. The downside is, without any structure around it (charters, time boxing, outline of goals, etc) I tend to dive into the minutia of some esoteric part, and I miss important, big things. On the opposite end of the spectrum you have detailed do-this-expect-that type test plans (or scripts if you want to go even further in that direction). These can be beneficial in verifying that a previous error no longer occurs in the exact way it did in the past (if you need to do that), or for giving brand new testers an idea of things to look for in testing. If all you ever do is follow plans though, you will not train your brain to search for anything ‘weird’. You will be repeatedly teaching yourself to only do what the paper says. Machines do that. Anyone else remember these existed?

Balance is the key. Use some sort of frame for your exploratory testing. Don’t just dive in totally free form; create enough structure around what you are doing to allow you to stop and take a breath once  . Coming back to reality will help keep you from getting stuck too far down in the details. If you are tasked with following a plan, stop every once in a while and look at the product. Look at it outside of the context of the expected results and try things. Give yourself a set amount of time and then go back to the plan.