Excellent testing requires skill, but heuristics give structure to that skill. Heuristics help us access our skills under pressure.

A heuristic is a fallible method of solving a problem or making a decision. Cem Kaner and I came to this definition based on an extensive search of papers and books across fifty years of psychology and engineering. Amazingly, we were not able to find a single coherent definition of heuristic in our research (The dictionaries are not much help, here) . Coleridge, Kant, and Polya have all written about heuristic reasoning. But we needed a definition that would bring the issues into focus.

There are two main issues: something that helps you solve a problem without being a guarantee. This immediately suggests the next issue: that heuristics must be applied sapiently (meaning with skill and care).

An excellent example of a heuristic is a hammer. Do you see how a hammer can help a carpenter solve a problem, but does not itself guarantee the solution? I like this example because it’s so easy to see that that a hammer may be critical to a skilled carpenter while being of little use to an unskilled lout who doesn’t know what to pound or how hard to pound it or when to stop pounding.

Heuristics do not replace skill. They don’t make skill unnecessary. But they can make skilled people more productive and reliable.

How Do You Tell if Consultants/Trainers Understand Heuristics?

I typically hear two reactions from rival testing consultants of other schools of thought (especially the Factory and Quality Control schools) that love “best practice” talk:

  1. “Oh yes, heuristics! That’s another name for ‘best practices’, right?”
  2. “Oh no, heuristics! That’s just a fancy name for ‘best practices’!”

Obviously both reactions miss the point. Even if these folks rename their ideas of what you should do to call them “heuristics,” they would be leaving out a key idea, which is that skill must rule methods. Talking about methods, focusing on methods, enshrining methods, is only sensible if humans are left in charge. The heuristic nature of engineering is the reason why a “best practice” is an absurdity. Seek not the perfect practice. Seek instead to practice your skills.

My friend Judah Mogilensky once commented that he thought my Heuristic Test Strategy Model was a lot like the Capability Maturity Model that he works with.

“Are people allowed to change or suspend the CMM whenever they see fit?” I asked.

“Oh no,” he replied.

“Then it isn’t being applied responsibly as a heuristic. It’s being treated like a law, or an oath, or something else that places itself above its subjects and binds them.”

So that’s how you tell. Look to see what’s driving things. People, or concepts? Fundamentally, “methodology” can’t control projects. Anytime you seem to see methods in charge, you are actually witnessing a project out of control, or else a project under covert control by shadowy masters.

When someone teaches you a way to solve a problem, check:

  • Do they teach you how to tell if it’s working?
  • Do they teach you how to tell if it’s going wrong?
  • Do they teach you heuristics for stopping?
  • Do they teach you heuristics for knowing when to apply it?
  • Do they compare it to alternative heuristics?
  • Do they show you why it works?
  • Do they help you understand when it probably works best?
  • Do they help you know how to re-design it, if needed?
  • Do they let you own it?
  • Do they ask you to practice it?
  • Do they tell stories about how it has failed?
  • Do they listen to you when you question or challenge it?
  • Do they praise you for questioning and challenging it?