ISTQB vs BBST

There is frequent debate about the value of the ISTQB certification and how it compares to certifications like the AST sponsored Black Box Software Testing courses. Having taken both ISTQB and BBST Foundation courses I feel ‘qualified’ to wade in on the debate. 

So what value does training and certification add?
A tester told me recently he saw no point doing training unless he got a certification at the end of it (thankfully he has not yet attended my course so I’m not taking offence…). I didn’t get chance to find out if he is someone who’s jaded by attending bad training that taught him nothing, or just someone who wants to quickly ‘prove’ to others he has knowledge rather than actually improve his skills and be able to demonstrate it. 

But this got me thinking that perhaps there is a gulf between the intention and perception of the value of each course. I have no doubt that both organisations set out with the intent of raising the standards of the testing profession, but has that been achieved?

In a recent training session we were discussing the use of equivalence partitioning and later in the day, white box testing. Two out of the ten attendees had to ask what the rest of us were talking about. So I did a quick poll as to who had taken their ISTQB and these were the only 2 who had not. 
Most interestingly, when I asked what was learned during ISTQB, the consensus from those who’ve taken the certificate is ‘the vocabulary’. However, everyone felt that in itself was worthwhile.  As our discussion showed, it gives testers from differing backgrounds, who work in an increasingly global environment, a common vocabulary on which to base discussions. This may allow time to be spent focusing on solutions rather than just trying to understand the basic definitions used. 

So, I asked some of my BBST class-mates what they felt they learned from BBST Foundations and the answers had much greater variation. They included the importance of context, the use of heuristics, the impossibility of complete test coverage and commonly for all responders, the importance of diversity in a team. On my course, the attendees varied from having 1 month to 24 years test experience… But each and every one of us learnt something from each other and about our strengths and weaknesses as a tester. We also learned about the importance of context when approaching a testing problem, via an exercise explicitly designed to demonstrate this.. 

BBST focuses on a wide range of testing skills. Every question can have many possible answers. If your justification is valid, then so is your answer. Instead of the ISTQB multiple choice model which can be marked by a computer, BBST requires you to develop original answers. It then asks you to mark other people’s work and then re-evaluate your own. You must demonstrate objectivity and critical thinking in order to pass.

It’s clear both courses have very different approaches. But do both courses add value to the testing world?

Lets consider the pass rate. Both courses pride themselves on their pass rate… But for very different reasons. ISTQB proudly state their 250,000+ certified testers. In contrast around 300 have passed BBST (as at Dec 2012). As one of both numbers I’m most proud of the exclusivity of being a BBST graduate, because I know the effort involved to pass. I have an instant level of respect for anyone who has completed the course. I’m sorry to say that an ISTQB graduate would have to find another way to gain respect as quickly.

Personally I prepared for ISTQB by self study, and my exam was taken in a Prometric test centre. As this may have limited the value I got from the course I spoke to others about their experience. They stated that the class sparked some good testing debate. Often people disagreed with the definitions given. However, after allowing some debate, the instructor would have to move the class on to cover the syllabus and would imply “whilst you have a valid point, you need to put aside your X years testing experience and learn the ISTQB syllabus in order to pass the exam.” 

As part of my research for this post, I revisited the ISTQB syllabus. I was surprised to find sections on tester psychology, principles of testing, a code of ethics and the importance of context in choosing the right test approach! Why then was this the exact opposite of the impression I came away from the exam with. All that stuck in my mind (and those of the people I’ve spoken to) was a limited range of techniques which should always be applied. Certainly nothing about the softer skills of testing and the importance of context. My suspicion is that these aspects are not included in the exam questions (perhaps because a computer is unable to mark the answers?) and therefore very little if any time is spent discussing them in class. 

In my opinion, every one can (and probably should within their first few years of testing) take the ISTQB foundation certificate. Not because it has a reputation for teaching people to be better testers but because it provides a common glossary of testing terms on which to base more intellectual testing discussions. It’s quick to take and easy to pass. My concern is that some employers see this as a ‘guarantee’ that they are employing a skilled tester. (and they might be, but this certification gives no guarantee)

But can / should everyone take the BBST course…?
I would like to say yes as as the content is invaluable for anyone who wants to be a better tester. But I suspect the pass rate will remain relatively low, for the main reason that it is a very hard course to complete and still harder to pass. It takes a commitment of 12+ hours a week for 4 weeks. This is not an easy commitment if you have a day job. The questions are hard, they take thought to solve. Not only do you have to formulate your own answers but you get judged on your ability to critique other people’s. If you don’t attempt to interact this way, you won’t pass. This takes dedication and a certain type of person who has commitment to learn and improve. Someone who chooses to take on a challenge and sticks with it. This certificate, to me, is one I will take seriously when I’m looking to employ testers.

So I’d like to open up this debate to you all. Have you taken either / both courses?  Or have you taken a different testing certification not mentioned here? What value did you get from them? Would you recommend to others and if so why?