One of the problems with going to a conference, at least for me, is that while I’m gone the things I’m supposed to do don’t go away. They build up. And so, once I get back, there’s a certain amount of digging out that I have to do. It’s the weekend now, though, and so I have a little time to look back on my notes from cast and try to capture some stuff on my blog. This post will be about Michael Bolton’s opening day keynote and is based on the notes I took during the keynote. I invite clarifications and correction in this post’s comments.

The day one morning keynote was to have been delivered by Cem Kaner but he was unable to attend CAST 2011 in person so Michael Bolton was asked to fill in. The talk started inauspiciously with Michaels laptop unable to connect to the projector. Paul Holland valiantly tried to get it working while Michael started his opening remarks but they just couldn’t get the laptop and projector working together so, after a comment about the dangers of a “scripted” presentation, Michael continued without the PowerPoint component of his talk. It was probably a better talk because of it.

Michael started his talk by recounting an episode of the CBC program “How to Think About Science” that talked about the controversies in science and in particular the dispute between Boyle and Hobbs about the validity of experiments and “matters of fact.” At issue was Hobbs’ questioning the validity of Boyle’s experiments around creating a vacuum using an air pump. Michael drew parallels between this controversy and issues that we face in testing such as the unreliability of tools and technology (given the materials available, Boyle’s air pump could not produce a perfect seal) and unrepresentative environments. In the context-driven school, we invite arguments such as Hobbs’ critique of Boyle’s experiments and we seriously consider those arguments’ validity because those challenges are a check on our work and our potential biases. Also, rather than seeing our job as reducing uncertainty, we should also recognize that we have a duty to work to eliminate unwarranted certainty.

He then went on to talk about the concept of testing “schools” and, in particular the Context Driven school and some attributes of it. In addition to inviting challenges to our work, these also include skepticism, recognition of the heuristic nature of our work, and acceptance of reality. The context-driven view sees testing more as a field science like anthropology than science done in a laboratory. We should all strive to avoid condemning without understanding. Michael talked about the value of “positive deviance,” safety language, and that a core skill for testers is “learn(ing) rapidly how to learn rapidly.”

A couple of other observations I jotted down in my notes:

  • Focus on reliability (of tests) can lead to poor testing
  • Theory and observation are both necessary