At last night’s Cambridge Tester Meetup, Karo talked about heuristics (slides here). After a brief introduction to the topic, she walked us through a couple of testing mnemonics:
We then split into two groups for an exercise. While the other group applied FCC CUTS VIDS to testing Karo’s kitchen – in fact, a schematic and floor plan of it – the group I was in took FEW HICCUPPS and a beer glass used at the 42nd Cambridge Beer Festival.
There’s plenty of pictures of the glass at #cbf42 but to give a quick description: it’s a pint glass with a loosely-themed Hitchiker’s Guide to the Galaxy/Beer Festival mashup logo (because it’s the 42nd festival, we assume) on one side and a Campaign for Real Ale (Camra) logo along with the festival name and dates on the other. It has calibrations for different amounts of beer, apparently in accordance with some kind of volume marking regulations which have a CE logo.
HICCUPPS is a set of consistency oracles and we agreed to use each of them as springboards for test ideas rather than receptacles of them, to avoid being constrained by whether an idea was “in” the category we happened to be discussing and risk losing it.
Here’s a selection of the ideas we came up with. I haven’t edited much, only to combine some overlapping items and lose some repetition and the notes I can’t understand this morning. We didn’t use the internet for the exercise, but I’ve looked up some references while writing this post and we could certainly use it for evidence and to inspire more questions.
- is the glass supplied at the festival always a pint glass, this shape, this size, of this manufacturing quality?
- is the logo in the same position, in the same proportions, in the same style across festivals? (e.g. compare images in the festival’s Flickr account)
- are the same volume measures always printed on it (pint, half-pint, third-of-a-pint)?
- does the glass always show the certification of volume using the CE volume mark?
- is there always a theme to the beer festival? Does it need to be reflected on the glass?
- is it important to the festival that there is continuity or consistency across festivals, glasses etc?
- is the festival logo on the glass intended to look this amateurish? (First impression: it’s like a student rag picture)
- There’s plenty of space on the glass, why squash the logo up in the way that has been done? (The mice have detail that’s hard to see)
- Would a simpler graphic design have been more striking?
- is it important that the measurements are accurate? (To what degree?)
- are the fonts chosen appropriate for the audience? (No comic sans!)
- is the use of colour appropriate? (The logo has to sit in front of many different colours of liquid)
- is there any relationship between Hitchikers Guide to the Galaxy and Camra? Are there any potential negative connotations that could be made?
- is the glass consistent with the festival beer listing booklet, posters, staff uniforms etc?
- is there a standard shape, size, material etc for beer glasses at festivals? How about at Camra festivals? Cambridge festivals?
- what about non-UK drinkers, what would they expect from a glass? In Britain, we still use imperial measurements but Cambridge is multicultural
- pubs often don’t have oversized beer glasses (as this one is, where the pint mark is below the top of the glass)
- how easy is it to clean vs similar products?
- what do similar kinds of events do about glasses? e.g. do wine festivals expect drinkers to use the same glass for red and white? Do they provide cleaning facilities for glasses? Is that part of this product?
- is the glass solid? Will it break easily if dropped? Is the flooring chosen to be gentle on dropped glasses?
- do Camra members have any expectations about the glass based on Camra conventions?
- we observed what we thought were injection moulding marks on our glass – would hand-made glass be expected by any attendees? (They are already connoisseurs to some extent by going to the festival.)
- are the volume markings correct?
- is the time and date information printed on the glass correct?
- is the vessel suitable for drinking beer from? Is it optimal? (What is the optimal glass for beer? cider? perry? soft drinks? Does it differ across beers?)
- is the glass dishwasher safe?
- would a glass from earlier beer festivals be honoured at this festival?
- what does other festival material say, show, suggest about this glass?
- does the glass alter the taste of its contents?
- does the logo imply some endorsement from Douglas Adams‘ estate? (Particularly since Adams was from Cambridge)
- is the glass built to last? (If so, last for what duration? The festival, life?)
- is this really the 42nd festival? (according to who?)
- is it easy to drink from? to hold? to pass between people (e.g. friends for trying a taste, to the bar staff?)
- is it stable when put down?
- should it be more tactile, e.g. with 3D logo on it?
- can this design of glass be stacked? is it stable when stacked?
- is it easy to fill, can the measures be seen by the bar staff?
- is it easy to carry multiple glasses (e.g. three in a triangle)
- is it unique (e.g. for collectors)
- it is robust?
- does it have appropriate thermal properties (e.g. help to keep cold beer cold?)
- is it safe (e.g. will it break into sharp shards when dropped?)
- do customers desire gender-specific glasses? (“Do you
want that in a lady’s glass?” )
- do customers want a glass that signifies no alcohol is in the drink? How about other kinds of specialist desires e.g. markings for gluten-free or vegetarian beer (is there such a thing, we asked? Yes.)
- how are the glasses packaged for transport? Are they space-efficient?
- are the production costs reasonable? affordable?
- what is the product here? (we have permitted ourselves to switch between the glass, use of the glass, the festival …)
- is the thickness of the glass appropriate, comfortable to drink from?
- do all of the instances of this glass at the festival look the same? Should they? To what tolerance?
- should there be half-pint glasses too?
- is the glass consistent with other aspects of branding?
- what is the Camra logo about? It’s looks like it has a lid. Is that intentional?
- do I want to drink out of it?
- is it obvious that it’s a receptable for liquids? For drinking from?
- is it suitable for display?
- does it look good in a collection of such glasses?
- can it be easily, safely, efficiently transported and stored?
- will the colours and other markings fade?
- what else could it be used for? (e.g. holding coins or pens, as a vase, watering flowers, magnifying glass … but this is a different testing exercise)
- does it chip easily?
- could you hurt people with it? (deliberately or not?)
- is the glass inert?
- can you stick it in your pocket when you need your hands free?
- is it compatible with devices for holding glasses (e.g. deckchairs, belts)
- what is the CE volume marking? Would we need to test it in some way (e.g. check that the manufacturer is licensed to use it?)
- are there hygiene standards for drinking vessels (e.g. certain grade, thickness, transparency of glass?)
- are there conventions, contractual agreements, regulations about using the name of Cambridge or Camra in association with events?
- does the festival have a license to sell beer?
- do the bar staff need licenses or training to serve beer?
- some brands of beer might require their product to be served in glasses branded for them?
- does the logo conform to copyright law (e.g. with Hitchikers Guide to the Galaxy images)
- does the glass fulfil the needs of the beer? (e.g. to have its head displayed, show bubbles, permit its colour to be appreciated, compared with others etc)
- are glasses required to be round? (if so, how round? Could it be square? elliptical?)
- what are common problems of any kind of branded product? branding wearing off, typos, correct copy etc
- glasses with handles are often a pain to fit into a cupboard
- the logo might not be obvious to people not in the intersection of Hitchikers and Camra fans
- explainability is a kind of testability heuristic
- the precise location of the festival isn’t given, only the town. Should it be precise?
- this is a pint glass to most Brits at least. To others it might just be a glass
- is it obvious a beer glass? Probably it is to those familiar with the conventions of such glasses
- does it obey the laws of physics?
- is it a practical object, or a collectors item?
- why does it have a wooden barrel in the logo when the beer at the festival no longer uses them?
- does the audience expect nostalgia?
- is it quintessentially English?