The following is an Experience Report submitted by Duncan Nisbet, who convened a peer workshop earlier this month, and was supported by the AST Grant Program.
Theme
The theme of the 4th North West Exploratory Workshop on Testing (NWEWT) was Testing the Future & this was the brief the attendees were given:
“The application of software is far reaching & it’s adoption into new & exciting domains is ever-accelerating.
As we find new & exciting problems to solve with software, how do we ensure that testing of the software remains appropriate & sufficient?
Consequently, we’d like you to explore the predictions of future applications of software & share with us your thoughts as to what the testing of that software may look like.
Of course, some of you are already living in the future & pushing the boundaries of software engineering! You may have experiences of how your testing strategy has had to be modified in order to remain effective & relevant – we’d love to hear about it!”
To help kickstart their thinking, we shared some interesting case studies:
- How would you test software that can “see” several minutes into the future?
- How could automation assist your testing Augmented & Mixed Reality?
- What would the SDLC of developing a nanorobot look like?
Sessions
Obviously, this isn’t the place for a write up of each session, but here are headlines of the sessions that got presented:
- Interesting pitfalls in Machine Learning
- How do you test a conversation you’re yet to have (re chatbots)
- The mobile phone is dead – how do you prepare for testing future tech?
- Tribal Ritual Assent on programmes pushing the technological boundaries (e.g. smart cities)
- Future Tech: for humans by humans – a discussion on a series of case studies where human bias has had a detrimental impact on the technology developed
- Rise of the Algorithms – case study of implementation of personalisation, conversational interfaces & prediction
- Case study of the challenges in testing biometric software
- Retrospective Futurespective – a look at the past to see if we’re making the same mistakes with future tech
- Security by stealth – an experience report into challenges facing test coaches / principles influencing team members. The report used the experience of helping team members start doing more security testing.
Attendees
Once again, we invited a diverse group of people in order to maximise chances of a wide range of lenses & opinions. This year, we had 15 job titles across 8 different domains represented in 1 room!
This increase in diversity was made possible by some attendees being empowered to invite others from their communities.
This experiment did lead to wider & deeper discussions as hoped for – once again, workshop conversations spilling over into lunch & dinner (we used the dinner card again).
We still need to get the invites out sooner!
Inspect & Adapt
There was a tremendous amount of feedback from last year’s NWEWT which we strived our best to incorporate into this year’s workshop.
Format
Building on some of the micro-experiments last year, we loosened the constraints on the format of the sessions.
This enabled attendees to be looser with their presentations which resulted in a couple of variations including the standard presentation / Q&A, open discussion based on a case study & an experiment into Ritual Assent (a spin on Ritual Dissent).
We also had a bigger room which meant we were able to have a change of scenery.
One half of the room was they classic U-shape, the other was soft furnishings & bean bags.
We only scratched the surface of what we could do with the space, but it was great to see some of the attendees keen to try something different.
We also had a mix of facilitators which was great as it gave everyone the opportunity to enjoy different facilitation styles. There’s also the benefit of all attendees getting the chance just to enjoy & contribute to a session without keeping order.
Venue
From the feedback of last year, we decided it was time to try a new venue. Some of the criteria for the new venue included
- Bigger room to facilitate a range of session formats (see above)
- Wider food choices for vegans
- Better food quality overall
- Better quality accommodation
Current thinking is we’ll keep the venue for next year.
Sponsorship
We did manage to secure extra sponsorship which meant that we could marginally reduce costs for some attendees.
Unfortunately, the cost of the venue was greater than last year, so some of that sponsorship was lost covering that price increase.
Summary
Overall it was another great session leaving us all exhausted by the end of the weekend for us all.
Key takeaways for me include:
- Starting Machine Learning isn’t as difficult as I thought it would be & that there are several tools to help facilitate that learning (like Kaggle)
- Voice Squatting is a thing!
- I need to do better research on future tech to help me focus my learnings
- Having focused opinions on open questions about large, complex technological solutions is HARD
- Autonomous vehicles can be hacked with stickers on the road & stickers on roadsigns
- Testers should understand the “black box” of the ML algorithm to help them test it.
- Our fingerprints change enough to not be recognised by biometric sensors when cold
- Engaged testers are just a small percentage of the engaged development community as a whole – what can we hope to change? How can we influence our wider communities?
- Companies like Tesla & Space X didn’t carry past cultures & behaviours with them – this enabled them to make dramatic changes to the domain they operate in What can we do force a paradigm shift in our domain? Can we even do that in our own domain?
- Start looking into the OWASP Juice Shop Project!
Thanks
Once again, I can’t give enough thanks to the attendees of this NWEWT – it wouldn’t be the same without them! Apologies for the lack of photo.
Andy Harding, Anik Kale, Ash Winter, Charlotte Smith, Chris Chant, Chris Thacker, Gwen Diagram, Ian Smith, Jitesh Gosai, Jonathon Wright, Leigh Rathbone, Lewis James, Matt Salmon, Paolo Addeo, Peter Langford, Swapna Bakshi.
Recent Comments