OZWST 2012 in Review

Attendees:

• Scott Barber (Facilitator)
• Anne-Marie Charrett (Content Owner)
• David Greenlees (Facilitator in training)
• Alessandra Moreira
• Andrew Dempster
• Aman Suri
• Kim Engel
• Mark Tolfts (day 2)
• Oliver Erlewein
• Richard Robinson

 

Day 1

As this was the first LAWST-style Peer CONFERence in Australia the majority of the attendees had not experienced the format.  Scott provided a brief overview, however preferred the ‘just in time’ approach to guiding people through it.  This helped as key items were explained directly preceding their use.  The more I see the ‘just in time’ format used, the more I like it!

Check-in was next.  Each attendee introduced themselves, their official title, what they ‘actually’ do (often quite different to what their title suggests – surprise!), and any distractions they were currently dealing with.  It’s great knowing what’s on people’s minds at that very point in time, and can help the Content Owner shape their approach.  For example, they will have picked one particular Experience Report (ER) to be first, however if the presenter of that ER is distracted by some sort of family issue, then this is known and can be accounted for; potentially changing the order.

Speaking of the Content Owner, Anne-Marie was up next to provide an overview of the theme and how it came about.  Our theme?  Influencing a Context-Driven Approach.  I felt a little bad at this point in time as I had pretty much determined the theme and ‘volunteered’ Anne-Marie to the Content Owner role.  Anne-Marie being awesome, was well prepared despite this.  Also, it is a theme close to Anne-Marie’s heart, which helped.  The main driver for this theme; to learn how to share the context-driven approach to testing, and gain the confidence to debate/converse with others that are of different mindsets.  I want to be clear here, this was not about learning how to make everyone follow our approach, because as we know well… there is never just one approach.  The last thing we want to do is tell the world, our way or the highway.  However we do want to be comfortable in telling the world, there is are different ways… let us show you how they work.

After our theme overview, it was time for ERs.  In keeping with the spirit of the Peer CONFERence IP Agreement, I won’t be going into great levels of detail while outlining each ER below.

ER 1 – Richard Robinson – Persuasive Progress Reporting Without Counting Test Cases

Richard has a knack for making me (and others) laugh.  His ER was no exception.  Richard told the story of a conversation between himself and his Project Manager; a conversation where Richard had a strong feeling that the subject of counting test cases would come up.  Sure enough, the subject was raised and Richard proceeded to walk us through the fascinating conversation that followed.

The clincher here was that Richard had prepared his response; as much as one can leading in to the unknown.

Richard took the attendees through his approach to testing the web services in the particular environment his ER related to.  It was impressive… and considering my current project is also using web services I took a lot away from it.

During Open Season (aka question time) the attendees discussed various aspects of Richard’s ER, most notable being how he went about reporting the approach that he has explained during his ER.

We then had a good amount of whiteboard action along with some ‘kung fu’ style movements from Richard while he was explaining the web service architecture in order to describe how he documented his test cases.  I think many of the attendees were a little unsure when Richard first mentioned documenting test cases during his ER, only to later discover that his process for doing so was ultimately lean, and very usable.

Through his ER he displayed not only an approach to testing, but also an approach to gathering and reporting metrics in a context-driven way.

ER 2 – David Greenlees – Death by Test Documentation

Then it was my turn.  I was nervous… not because of my ER, but because I wanted so much for OZWST to be successful.  I had put many hours of effort into making it happen, and I was desperate for all attendees to learn something and enjoy the experience.  It was too early at this stage for me to be comfortable that OZWST had achieved my goals.

Despite these nerves I thoroughly enjoy giving my ER.  I spoke of my years of test documentation ‘battles’.  The fight against many templates with pre-filled information to the point that only a find and replace of the project name was required.

I didn’t spend a lot of time talking about what I feel is required/not required in a template, but more about how I approached the ‘battles’.  I had tried a few different ways of explaining different approaches to test documentation over the years; the primary goal being to eliminate waste, and prompt the use of people’s brains (i.e. no pre-fill of information).  Some of these ways had limited success at best… needless to say it was great listening to the attendee’s suggestions and their questions which prompted me to rethink various aspects of my approach.

I left my ER with new goals in this area of my work, and plenty of valuable ammunition to get me started!

Tour

After lunch Andrew and Schneider Electric (our wonderful and very up to date venue for the event) were kind enough to offer a tour of the hardware testing section.  It was very interesting to see how they test various components and Richard managed to ask many question in relation to standards and how Schneider Electric feels they could be improved.  He had us all laughing once again!  I think many of us left the tour wanting to test hardware.  Then back to the ERs…

ER 3 – Andrew Dempster – Context-Driven Testing in a Large Multinational Organisation

Andrew’s ER was amazing.  To me; a tale of success.  To Andrew; a never ending tale of some success and a LOT of hard work.  I have known Andrew for a good period of time now, and have discussed his approach over a few beers now and then… therefore I know how much hard work he has put in to get his test team where they are today.  After his ER, I think all the other attendees also shared my knowledge in relation to his hard work.

Andrew works for a large multinational company with many dispersed development and test teams.  His being the only team approaching testing in a context-driven manner, and utilising Session Based Testing with some fantastic results.

He spoke of his initial introduction to the company along with another of the attendees of OZWST (Mark), and of how his previous testing experience (factory school approach) did not work in the new environment for many reasons.  This prompted him to begin research into context-driven testing and at one stage even made the call out to James Bach to come and teach his Rapid Software Testing class.

This was a truly fascinating story (even if I had heard parts of it before), and I believe the clincher here was that it was a success!  No matter how you look at the journey Andrew has taken over the years… it’s ended with a successful test team who are getting great results and testing according to their context.  There is no doubt that Andrew will be up against it for years to come, but he can take stock of some great achievements to date.

There was time for some great Open Season discussion prior to check-outs and announcements from the organisers.

This brought us to the end of day 1.  Through the check-out it was clear that all attendees were still energised, which was a great result.  For those that have attended a Peer CONFERence, you would know that mental exhaustion is not uncommon.

Also a bonus for the extra energy in the room was the fact that an OZWST dinner had been arranged at the beautiful Andre’s Cucina & Polenta Bar in the east end of Adelaide CBD.

There was great food, great conversation, napkins being drawn on, and some candles!  That’s right; day 1 was also Anne-Marie and Kim’s birthday!  Special deserts with candles and OZWST attendees singing Happy Birthday… who knew testers could sing?

Day 2

Attendees arrived just on time for day 2 after a slight taxi/attendee ‘mix up’.  I won’t go into detail in order to avoid incriminating a select few!

The energy was still good and check-in proved that day 1 was a success.  Attendees were ready for another day of full on discussion, learning, and were keen to get straight to more ERs.

ER 4 – Aman Suri – The Only Context-Driven Tester in a Multinational Project

Aman spoke of his very first project working for his current employer.  As the title suggests, he was the only context-driven tester with all other testers following a factory approach.  He was responsible for testing one component of a system made up of seven in total.  From the outset he was advised to only focus on his component, but very quickly realised that in order to test it well he would need it to interact with the other six.  He wanted to ensure the ‘real world’ scenario for use would be successful.  The attendees agreed that this big picture view from Aman was exactly what was required in this situation.

This approach was not the norm for these projects.  Aman explained that this would be termed System Testing, and that was scheduled for later in the project, yet he felt that it was required earlier in order to find important bugs fast (music to our ears!).

Session Based Testing was utilised by Aman, and the usual requests for test cases/scripts were made by the other Test Managers and Project Managers.  Aman was steadfast in his approach and instead provided full access to his session sheets and dashboard reporting.

As a result of Aman’s approach, and the resulting communication with the development team, his component of the system to this day remains in good quality while the other components are still holding up the implementation of the overall system, for various and all too obvious reasons.

Another success story for context-driven testing; this has provided both Aman and his team with valuable experiences that they can demonstrate in the future when influencing the context-driven approach.

ER 5 – Kim Engel – Context-Driven Testing in a Bureaucracy

Kim’s ER was one of those that need to stay within the four walls of the Peer CONFERence.  She displayed great courage in talking about her ‘battles’ for a better way of testing.  Kim’s ER highlighted to many of the attendees that the small wins can actually be just as important as the big wins.  She took the attendees through a journey of various bureaucratic conversations, how she handled them, and what results she felt that she gained throughout.  There was talk of test case counts, developer co-location, session based testing, exploratory testing, and even some stealth testing as well.

Of particular note was her approach to test cases for use by UAT testers.  Instead of lengthy step by step scripts she developed a checklist approach which guided the testers via idea generating dot points.  This helped the testers avoid the age old problem of inattentional blindness.  This will be a lasting legacy, and an example of a very valuable small win.

Kim also highlighted to attendees two points that she was steadfast in opposing.  These were capturing screen shots for every single test, and traceability back to requirements.  Kim felt that capturing screen shots for all tests was not required as she trusted her testers, and felt that others should too.  Of course if any problems were encountered then these would be captured.  The way the scenario was described I agreed that this would be an unnecessary overhead.  As for linking test cases back to requirements… why would you when the requirements were not completed!

Role Play

This session was originally suggested to me by Mark during the initial planning for the event.  I had intended for this to be a ‘fun’ session along with a valuable one in terms of learning how to handle the many questions we context-driven testers get asked.

Scott and Anne-Marie (our rock star testers) took to the hot seats at the front of the room while attendees threw curly questions their way in relation to many factory approach tasks.

I got the ball rolling by welcoming both Scott and Anne-Marie to my wonderful consultancy called ‘ISO Nine Thousand & Something’.  I had employed them to head into clients and implement our new software testing standard.  What followed was a brilliant conversation in relation to standards and why we do/don’t need them.  Scott provided wonderful analogies which provoked a lot of thought… needless to say that I ended up changing the name of my consultancy!

Richard then requested their help in writing a test plan, because the process required it.  There was back and forth while Scott and Anne-Marie both established context via questioning Richard.  This resulted in a suggestion from Anne-Marie to bring all the key players into a room so that she could whiteboard the plan as an initial step while Richard did his best to play ‘dumb’.

What followed was fascinating.  Good interaction from attendees in seeking answers to the many questions and problems they face when trying to test in the best possible way; context-driven.

It was fun, of that there is no doubt.  It was also valuable.  So thank you to Mark for the suggestion.

ER 6 – Oliver Erlewein – Learning From History… Or Not

I loved Oliver’s ER.  He took a different approach and actually read out a blog post that he has had sitting in draft for a while now.  It will end up being published, so I won’t go into detail.  Needless to say, I suggest you read it when it’s released.

His post took us through the many problems he sees in our industry, at which point he stopped to allow for Open Season.  These problems produced some passion in the room and discussion threads were flying everywhere.  Once the threads had dried up I facilitated a vote to see if the attendees wanted Oliver to continue reading, which would cover his thoughts on how to combat the problems he had read about previously… a unanimous thumbs up!

So Oliver continued to read his post and once finished a second Open Season was facilitated which equalled more passion from the room!

As stated previously I urge you to read the post once it’s published.

ER 7 – Alessandra Moreira – Winning Small Battles Before We Win the War

We were now nearing the close of Day 2.  There was just enough time for one last ER.  Alessandra took the ‘stage’ as she was particularly keen to present her story, and what a fine story it was.

Alessandra spoke of being the only context-driven tester in her small team of three, an all too common scenario it would seem, in a financial services environment (highly risk adverse) working with trading systems where every second counts!  She quickly established that she was going to need to win the small battles in order to demonstrate how effective exploratory testing and a context-driven approach can be.

She discovered that the existing test cases where highly detailed, with the number of steps anywhere between 50 and 80.  Some jaws in the room did drop when she stated that fact!  The other two testers in her team were described as ‘switched on’ and knew the systems well as they had been there for about a year.  To explain context-driven testing to her team Alessandra took a ‘coffee shop’ approach which was quite effective.  Taking her testers out of the working environment helped them to concentrate and value the lessons more so than otherwise.  There were several agreeable nods around the room, so I would suggest many of the other attendees have been successful with this approach on previous occasions.

Focusing on those small wins, Alessandra described some of her achievements over the two years that she worked for the team.  Test cases decreased in size considerably, and more importantly the testers actually got excited about testing!  Also, she managed to move the test team so they were co-located with the development team which assisted in reducing the existing ‘us versus them’ mentality.

Another notable point from Alessandra’s ER was ET Friday’s.  As a compromise with her Manager it was agreed that three hours each Friday would be dedicated to exploratory testing.  All three would work in a room so that communication was easier.  They would begin with a quick mind-mapping session in order to plan their session and break up the areas of the system.  The results of each session were reported back to her Manager, and the testers enjoyed the experience as they were no longer bored!

Alessandra’s overall goal for this assignment was to achieve a full exploratory testing approach, and as this didn’t happen she felt that she had failed.  Upon reflection, the small wins she had did actually amount to a success.

During open season it was discovered that Scott has a particular ‘interest’ in how UAT is approached within our industry.  There were some fireworks, however facilitation was solid and all attendees maintained the required level of control.  This is one of the many reasons that these events work so well.  It’s a ‘safe’ environment where rules are in place to ensure things don’t get out of control.

Next on the list was wrap up and check-out for day 2.  As mentioned earlier I had been nervous for most of the workshop as I really wanted it to go well.  Listening to the attendees check-out, hearing their praise, and what they had learned; the nerves vanished.  OZWST was a success!

I would like to thank all attendees for their willingness to share such personal stories.  The courage in the room was amazing.

I also need to give special thanks the following:

• Scott Barber – Our very own testing rock star direct from the US of A!  Scott was more than happy to change his travel plans to accommodate OZWST and did a stellar job of facilitating and training me in that fine art.  I look forward to collaborating with him more in the future.
• Anne-Marie Charrett – Content Owner extraordinaire!  As mentioned, I threw this role at Anne-Marie and she caught it with both hands.  Having not done it before, she nailed it.
• Andrew Dempster – My co-organiser!  Andrew did a considerable amount of work on OZWST.  He took care of all logistics, and executed the sign making with precision.  Andrew will be a great help in organising OZWST 2.
• Association for Software Testing – Via their Grant Program the AST supported OZWST and without them I would have been considerably out of pocket (if I was able to manage it at all).  Matt Heusser and Michael Larsen were extremely easy to work with and the turnaround time for the grant was outstanding.

Planning is already underway for OZWST 2, and we’re currently talking with other testing rock stars that may be able to take part.  Thank you once again to all involved.  A truly wonderful and engaging event which strives to change the face of testing down under!

David Greenlees

On behalf of the OZWST Committee

http://ozwst.wordpress.com/