The following comment was posted to Twitter yesterday by Aaron Hodder, and I felt compelled not just to retweet it, but to write a follow up about it.
“People that conflate being anti wasteful documentation with anti documentation frustrate me.”
I think this is an important area for us to talk about. This is where the pendulum, I believe has swing too far in both directions during my career. I remember very well the 1990s, and having to use ISO-9001 standards for writing test plans. For those not familiar with this approach, the idea was that you had a functional specification, and that functional specification had an accompanying test plan. In most cases, that test plan mapped exactly to that functional specification. We often referred to this as “the sideways spec”. That was a joking term we used to basically say “we took the spec, and we added words like confirm or verify to each statement.” If you think I’m kidding, I assure you I’m not. It wasn’t exactly that way, but it was very close. I remember all too well writing a number of detailed test plans, trying to be as specific as possible, only to have it turned back to me with “it’s not detailed enough.” When I finally figured out what they really meant was “just copy the spec”, I dutifully followed. It made my employers happy, but it did not result in better testing. In fact, I think it’s safe to say it resulted in worse testing, because we rarely followed what we had written. Qe did what we felt we had to do with the time that we had, and the document existed to cover our butts.
Fast forward now a couple of decades, and we are in the midst of the “Agile age”. In this Agile world, we believe in not providing lots of “needless documentation”. In many environments, this translates to “no documentation” or “no spec” outside of what appears on the Kanban board or Scrum tracker. A lot is left to the imagination. As a tester, this can be a good thing. It allows me the ability to open up different aspects, and go in directions that are not specifically scripted. That’s the good part.
The bad part is, because there’s sparse documentation, we don’t necessarily explore all the potential avenues because we just don’t know what they are. Often, I create my own checklists and add ideas of where I looked, and in the past I’ve received replies like “You’re putting too much in here, you don’t need to do that, this is overkill.” I think it’s important for us to differentiate between not overthinking and over planning, and making sure that we are giving enough information and providing enough details to be successful. It’s never going to be perfect. The idea is that we communicate, we talk, we don’t just throw documents at each other. We have to make sure that we have communicated enough, and that we really do understand what needs to happen.
I’m a fan of the “Three Amigos” model. Early in the story, preferably at the very beginning, three people come together; the product owner, the programmer and the tester. That is where these details can be discussed and hammered out. I do not expect a full spec or implementation at this point, but it is important that everybody that comes to this meeting share their concerns, considerations and questions. There’s a good chance we still might miss a number of things, because we didn’t take the time here to talk out what could happen. Is it possible to overdo it? Perhaps, if we’re getting bogged down in minutia and details, but I don’t think it is a bad thing to press for real questions such as “Where is this product going to be used? What are the permutations that we need to consider? What subsystems might this also affect?” If we don’t have the answer right then and there, we still have the ability to say “Oh, yeah that’s important. Let’s make sure that we document that.”
There’s no question, I prefer this more nimble and agile method of developing software to yesteryear. I would really rather not go back to what I had to do in the 90s. However, even in our trying to be lean, and in our quest for a minimal viable products, let’s be sure we are also communicating effectively about what our products need to be doing. My guess is, the overall quality of what comes out the other end will be much better for the effort.