Workshop on Heuristic & Exploratory Techniques (WHET)

Over 30 years ago, Kaner coined the phrase “exploratory testing” as a contrast to traditional, scripted software testing.  Since then, the approach has been alternately praised and vilified. Competing mythologies have developed about the risks, benefits (even the morality) of exploration. And — some people have gotten pretty good at it.

Over the years, the concept has evolved and been refined. We currently define software testing as a process of technical investigation, an empirical study of the product under test with the goal of exposing quality-relating information about it. (Validation and verification both fit within this definition.) We define EXPLORATORY testing as an ongoing interaction of three activities: learning, test design, and test execution. Throughout the testing project, the explorer researches the product, its stakeholders and their objectives, its market, competition, capabilities, platform, the products it interacts with, and risks associated with any of these.Based on this information, the explorer designs tests, runs them, and learns more about the product and how it should be tested.

Notice that what is different between exploratory and more traditional scripted testing is not the collection of techniques used. It is the cognitive involvement of the tester. In a scripted testing situation, the tester runs a test that applies to a test technique because a test planner previously decided that this was the right type of test and this was the appropriate test data. In an exploratory situation, the tester is the test planner. There might be a testing strategist, a coach whoguides the tester, but the tester picks the technique to use now based on her assessment at this time of what will most likely yield the most valuable information. The explorer might assess risks and develop a plan in advance, might prepare data in advance, but at every moment of testing, the explorer is responsible for deciding whether the plan for today’s work is still the right plan, whether and how to use the data, what new information is needed and what is the best way to get it.

Resources

 

WHET 4

  • Topic: Boundary Testing
  • Date: July 7 -8, 2007
  • Location: Seattle, WA

WHET 3

  • Topic: Task Analysis
  • Date: May 19-21, 2006
  • Location: Melbourne, Florida
  • Co-Hosts: Cem Kaner & James Bach
  • Facilitator: Scott Barber

We think it’s time for a task analysis. What do skilled exploratory testers do? How can testers get better at these tasks?

Participants in this meeting will be expected to come prepared, not just with ideas and experience reports about exploratory work. Also with some background reading and thinking about task analysis. We specifically recommend Jonassen, Tessmer & Hannum’s TASK ANALYSIS METHODS FOR INSTRUCTIONAL DESIGN.

To achieve common ground for this meeting, we ask that all participants read Jonassen et al before coming to the meeting, and write some notes, ready to be shared with the other participants, that apply ideas from the book to testing or on how the analyses described in this book can be applied to improve our understanding of testing. If you are not willing to commit to reading this book and thinking about its applications before coming to the meeting, please do not apply to come.

There are many other interesting and relevant books that might give you related ideas or help you understand some of the ideas in Jonassen et. al.A few examples are Gause & Weinberg’s EXPLORING REQUIREMENTS: QUALITY BEFORE DESIGN, Schraagen, Chipman & Shalin’s COGNITIVE TASK ANALYSIS, Hackos & Redish’s USER AND TASK ANALYSIS FOR INTERFACE DESIGN, Cooper & Reimann’s ABOUT FACE 2.0: THE ESSENTIALS OF INTERACTION DESIGN, Annett & Stanton’s TASK ANALYSIS, and Carroll’s SCENARIO-BASED DESIGN: ENVISIONING WORK AND TECHNOLOGY IN SYSTEM DEVELOPMENT. Familiarity with some of these books, especially familiarity accompanied by thinking about how what’s in the book could help us understand how to examine what testers do, what challenges testers face, what skills testers develop, what knowledge testers seek (etc.) will be helpful, useful in the meeting. But we will run the meeting on the assumption that you have read Jonassen.

 

WHET 2

  • Topic: Demonstrating Exploratory Techniques
  • Date: March 2001
  • Location: Front Royal, VA
  • Co-Hosts: Cem Kaner & James Bach

    WHET 1

    • Topic: Paired Exploratory Testing
    • Date: March 2000
    • Location: Front Royal, VA
    • Co-Hosts: Cem Kaner & James Bach