On Linkedin David Morgan started an interesting titled: “ISO/IEC/IEEE 29119 – why the fear and opprobrium“. In this discussion Cor van Rhijn asked me this question:
“@Huib,
your comment gives the impression that you do not believe in standards,
Please enlighten me and let me know what are the DISadvantages to standards.
Personally I am a strong believer in standards, given that they are applied in a matter that is suitable to the environment and the problem (so with regards to complexity, size and risk) and that they should be used as a guideline and that issues that are not relevant should be omitted, due to your consideration and specific situation.
In that respect I would like to refer to the IEEE standards for software test documentation where this idea is phrased explicitly in the text of the standard.“
Much has been said about ISO29119 the last weeks. For some background, please have a look at the many things said online in my collection of resources on the controversy.
So what is wrong with this ISO 29119 standard?
- The standard is not available publicly. How can I comply to or even discuss a standard that is not publicly available?
- ISO is an commercial organisation. The standard is a form of “rent-seeking“. One form of rent-seeking is using regulations or standards in order to create or manipulate a market for consulting, training, and certification.
- The standard embodies a document heavy test process which is unnecessary and therefor in many situations waste. Didn’t history show us that documentation and processes are important but that there are more important things we should consider?
- The standard doesn’t speak about the most important thing in testing: skills! The word skill is used 8 times in the first three parts of the standard (270 pages!) and not once it has been made clear WHICH skills are needed. Only that you need to get the right skills to do the job.
- There is much wrong with the content. For instance: the writers don’t understand exploratory testing AT ALL. I wish I could quote the standard, but it is copyrighted. For those who have access to it, here are some examples: have a look at the definition on page 7 in part 1, the advantages and disadvantages of scripted and unscripted testing on page 33 in part 1 or what the standard says about test execution on page 37 in part 2.
- I think there is no need for this standard. I have gone through the arguments used in favour of this standard in the slides of a talk by Stuart Reid (convener of ISO JTC1/SC7 WG26 (Software Testing) developing the new ISO 29119 Software Testing standard) held at SIGIST in 2013 and Belgium Testing Days in 2014:
“Confidence in products”? Sure, with a product standard maybe. But testing is not a product or a manufactory process! “Safety from liability”? So this standard is to cover my ass? Remember that a well designed process badly executed will still result in bad products. Guidelines and no “best practice”? I wish it would, but practice shows that these kind of standards become mandatory and best practice very soon…
Common terminology is dangerous. Read Michael Bolton posts about it here and here. To be able to truly understand each other, we need to ask questions and discuss in depth. Shallow agreement about a definition will result in problems. Professional qualifications and certification schemes? We have those and they didn’t help, did they? Benchmarks of “good industry practice” are context dependant. The purpose of a standard is to describe stuff context-free. So how can a standard be used as a benchmark? Ah! Best practice after all?
Who is demanding this standard? And please tell me why they want it. There will always be conflicts in definitions and processes. We NEED different processes to do our job well in the many different contexts we work in. A baseline for the testing discipline? Really? Without mentioning any context? What the current industry practice is lacking are skills! We need more excellent testers. The only way to become excellent is to learn, practice and get feedback from mentors and peers. That is how it works. Buyers are unclear what good test practice is? How does that work with selecting a doctor or a professional soccer player? Would you look at their certifications and standards used or is there something else you would do?
I do believe in standards. I am very happy that there are standards: mostly standards for products, not processes. Testing is a performance and not a pile of documents and a process you can standardise. I think there is a very different process needed to test a space shuttle, a website and a computer chip producing machine.
I wish that standards would be guidelines, but reality shows standards become mandatory often. This post by Pradeep Soundararajan gives you some examples. That is why I think this standard should be stopped.
Finally, let’s have a look at what the ISO claims on the “http://www.softwaretestingstandard.org/” website:
“ISO/IEC/IEEE 29119 Software Testing is an internationally agreed set of standards for software testing that can be used within any software development life cycle or organisation. By implementing these standards, you will be adopting the only internationally-recognised and agreed standards for software testing, which will provide your organisation with a high-quality approach to testing that can be communicated throughout the world. ”.
Really? I think it is simply not true. First of all, since the petition is signed by hundreds of people from all over the world and over 30 people blogged about it, I guess the standard is not really internationally agreed. And second: how will it provide my (clients) organisation with a high-quality approach? Again: the quality of any approach lies in the skills and the mindset of the people doing the actual work.
I think this standard is wrong and I signed the petition and the manifesto. I urge you to do the same.
Recent Comments