The quarterly Nashville Quality Assurance meetup happened this afternoon. The topic for todays talk was steps and organization can take to move from a primarily manual shop to a primarily automated shop. These sort of talks usually give me a pretty visceral reaction and the pages of (biased) notes I tool today probably reflect that pretty accurately.
I wanted to talk to the speaker but dominating his time didn’t seem appropriate for this sort of venue. So, as a result of that I’m writing this.
Here are some of the things that bugged me the most:
Not sharing your story
I have a hard time respecting talks that make sweeping generalizations and endorse specific ways of doing things without sharing failures, detailed context, and lessons learned along the way. This style of presentation gives me the impression that they are trying to share some sort of gospel. Phrases such as “this is the natural spot for X”, “This is how to get things right”, and “center of excellence” really drive that feeling home for me.
Not mentioning the gruesome details
The fellas that did this talk come from a Nashville company called Assurion. This company employs a varitable army of testers and developers to maintain the create the product. I feel like the fact that a large staff is needed to create and maintain this sort of high-volume automation was conveniently ignored. DSLs don’t magically get created, software tooling doesn’t happen magically. All of this takes significant time and development resources.
Rehashing folk wisdom
The folk wisdom is bullshit…whew I feel better now.
Here are some of the classics from today:
1 – X% of your tests should be automated.
Which percentage? Why? Why not the other percentage?
2 – Automation makes testing faster, more scalable, and able to be performed unattended.
No….just no. There is development time, maintenance time on the scripts, product, and framework, investigation for test failures, false positives, timing issues….the list goes on.
3 – The people writing code all day to make this sort of automation happen are testers.
The best I can give you here is a maybe. *Some* testers have the fluency and care enough about programming to do this. What I have seen most often though is a programmer(s) with heavy interest in tool smithing. Either way, these people will be writing code for the majority of their day be it test code or code to facilitate that.
4 – Test repeatability creates return on investment.
What is the investment? How did you measure that? What is the return? How did you measure that? Unwillingness to talk about this in terms of value as opposed to cost is baffling. If repeatability creates ROI, does that mean a test that is run once and never repeated has no return on investment?
5 – X scripts should be created per day per person
The reasoning for this aside from time accounting escapes me. I suspect it is linked to the fascination with ROI and ignoring value.
Tool fetishism
Spending time talking about all the tools you have used, all the tools you currently use, how much money you saved by switching from X to Y is uninteresting. Especially when you don’t frame the talk around what problems you were solving. Especially when you don’t show any examples of what you are actually working on.
This is a lot of stuff so I feel the need to mention a couple things. I don’t hate automation, it can be really useful in some situations. I really like the phrase tool-aided testing that has been gaining some traction lately in the community. This encapsulates the sort of automation described above as well as any other action we perform in which we use tools to amplify our ability to do something.
The speakers were charismatic and seemed like nice people. There, I said something nice 
Recent Comments