I had the pleasure of hosting the second Online Summit, delivered by Software Test Professionals: Achieving Business Value With Test Automation. The online summit format consists of 3 sessions each for 3 consecutive days. The sessions for this summit were:
- Software Test Automation, Where’s the Value? – Scott Barber
- Achieving Value with Automated Acceptance Tests – Linda Hayes
- Developer Level Automation Value from “Velcro” to “Velocity” – Rob Sabourin
- The Simple and Powerful Build Verification Tests (BVT) – Alan Page
- Functional Automation is the Gateway Drug to Continuous Delivery – Adam Goucher
- Test Automation at the API Layer…Why Bother? – BJ Rollison
- Experiences with Adding Value Through Test Automation – Mark Fewster
- Test Automation Misconceptions and Money Pits – Hans Buwalda
- Top Tips for Addressing the Challenge – Scott Barber
- Speaker Panel – Scott Barber, Linda Hayes, Alan Page, Adam Goucher, Mark Fewster & Guest Panelist Griffin Jones
One of my duties as host was to try to summarize the most valuable nuggets of information from across all of the presentations into a “top tips” list. This is what I came up with:
- Not every tip is specific to test automation
- Several tips are points made by more than one presenter
- Like every good “Top 10 List”, tips are presented in reverse order (according to me)
- I’ve paraphrased many of the tips to make them “quippy” 🙂
Tip #10: Pure record/playback => FAIL
- Record/playback is simple, seductive, and is (in most cases) not the most valuable answer to a question worth answering.
Tip #9: Learn automation in low risk situations
- Do not start out by automating tests against production
- Do get your skills “up to snuff” before making promises your automation can’t keep
Tip #8: Forget ROI, think Cost/Value
- Provide Business Value
- The ROI of testing can’t be a positive number (unless your business product is testing services)
- Determine what value automation can *reasonably* provide
- Estimate the cost of providing that value
- Ask “is that value worth the cost?”
Tip #7: Valuable automation is trustworthy
- To be trustworthy, automation must not be excessively complex
- Diligently avoid false positives.
- Err on the side of false negatives.
Tip #6: Design Build Verification Tests (BVTs) to tell you if additional testing is valuable at this time
- Do not try to design BVTs to tell you whether or not the build is release ready
Tip #5: Test at different levels to accomplish different goals — use the tools that work/makes sense
- Unit, Object, Integration, API, and/or GUI
- This is true for manual, automated, or other forms of testing
Tip #4: In the land of automation, test design is king
- Brainlessly automating manual tests typically leads to a whole lot of NO VALUE
- Modularity
- Executable Requirements, Keywords, Classes, & Actions
- Test Data, Test Data, Test Data
Tip #3: Collaborate … Just do it!
- There is no wall between developers and testers, there are only politics.
- Reject politics and collaborate to accomplish the mission
- Testing is not a union that prohibits others from testing or testers from doing other things.
- Enable the right people, with the right skills to add the most testing value at the right time in the most efficient/effective manner
Tip #2: Automate things that are better suited for computers to do than humans — only if they have value
- Application deployment is a great candidate for automation – whether or not any automated testing is included.
Tip #1: Do not automate what requires human judgement.
- Just don’t.
- ‘nuff said.
—
Scott Barber
Chief Technologist, PerfTestPlus, Inc.
About.me
Co-Author, Performance Testing Guidance for Web Applications
Author, Web Load Testing for Dummies
Contri
buting Author, Beautiful Testing, and How To Reduce the Cost of Testing
“If you can see it in your mind…
you will find it in your life.”
Recent Comments