What Looks Like Is Wrong May Not Be What's Wrong: Testing Lessons From House, M.D.

So a few weeks ago, I was trying to figure something out.  I was getting weird error messages and strange results and generally odd behaviors when trying to run a test.  Now, the odd part was, the behaviors shown and the error messages and the data stored in the DB simply did not, well, match. 

There was no way those pieces could all go together.  What could be going on?  What could be causing these really unusual symptoms? 

Now, I don’t know what anyone else does in these situations.  I know that the method I use for tracking down problems like this can best be described as “Housian” – Well, maybe “House-ian.”

Yes, I turn to that ever so patient and thoughtful, gentle tempered example for all testers, the title character of “House M.D.” Gregory House.  The cuddly, lovable, always cheerful soul dispensing folksy wisdom in an gentle, kind way.

OK – If you ever watched the show, you know I’m, well, fibbing a bit.  He isn’t most of those things, usually. 

What he is, is methodical.  He may not be the gentlest soul, but there is something about him.  You see, when problems simply can’t co-exist – I think of it as two objects not being able to share the same space.  I sort of learned that once upon a time.  The core question is what are you going to do about it?  How are you going to find the real problem that is triggering the first in the observable problems? 

House uses a team of people to bounce competing ideas against.  That can work in some circumstances, like when you have a short time to figure out what is wrong with a critically ill patient on television, and a commercial break is coming up.  What I do like is to keep ideas flowing.

Sometimes I find myself looking at a symptom or two and trying to find a relationship.  Is one causing the other?  Can the “real” problem be something less than obvious?  Maybe another set of problems is masking the real issue?  Maybe the real issue has nothing at all to do with what I am seeing? 

Yeah, I know.  It kinda depends on what the symptoms are, right? 

So, with this particular set of problems, I found myself thinking, “What would House do?”  Other than belittling all the ideas from the Greek Chorus of supporting doctors, a common tactic is to focus on something that I can control – one symptom or set of symptoms and carry on with that.  Maybe I’ll find something related to what the core problem is. 

So, I tired that.  Nope.  Nothing appeared to make sense.  Back to square one.

Then something dawned on me.  Maybe the clue to the real cause was being masked in the flood of stuff in the logs.  Maybe I was ignoring the one piece of information I needed.

So I took a look at the logs again.  (Folks who watch the show will see this as the “Hey, look at this little spot in this easily ignored place on the patient’s body” moment.)

Bingo – there it was.  The one little error message in the midst of spectacular dumps of… stuff.  There was the clue I was ignoring all along. 

Thanks, Doctor House.  We were able to save the patient.  Umm, project.