Jo Reid, Creative Directer of Calvium has worked for years with artists and creatives on creating technology driven experiences, so we asked her to join us at the Theatre Sandbox community day and share some of her insight into how to effectively develop, test and refine ideas:

Try things out in the actual space – testing things theoretically is not good enough: When HP Labs worked with Duncan Speakman on Schminky they started off using ultrasonic positioning to connect people in Watershed’s cafe. Early testing discovered that the clinking of metal cutlery interfered with the technology, if they hadn’t found this out early it could have been a lengthy and expensive dead end.

Appreciate your collaborators different strengths: Treat working with technologists as a mutual learning opportunity - by working in partnership you can come out with something greater than the sum of your skills.

Understand the context of what happens in the space you are using – what kind of people are in the space ? How are they using it already? Walk the space and take notes/pictures. The most compelling experiences acknowledge the space/situation – otherwise what is the point in making use of the outside world?

Decide the mechanics of the piece: What are the cues that make people look up/around/at something specific? Again, use technology to augment spaces rather than distract from them.

How do you get feedback from users? Test and then ask them about the usability of the piece, how enjoyable it was, what they understood, what knitted together. Some techniques:

• Reflection after the event (surveys and interviews)
• Passive sampling sampling – data logging and video
• In the moment sampling – observational studies and listening to what people say as they experience the project
• Longitudinal study: diary and ethnographic studies

On the subject of questionnaires - we got conflicting results to the theatre sandbox workshop evaluation and learnt quickly about survey design: if you are asking people how much they enjoyed something – it is natural that the ‘enjoyed a lot’ tick box should go on the right hand side of the paper. Otherwise people will ignore the actual heading and default to this.

Testing can be done in project phases. Isolate a core piece or moment or effect and test on its own. This doesn’t always need to be done with complex technology: ‘The Wizard of Oz’ effect means faking the effect you want before you build the technology. You can also mock stuff up with paper – draw out the options/screens/instructions and try them out with people to tease out usability issues.

Body storming: if your piece involves getting people to move around a physical space, make scale representational maps that people can explore and annotate with ideas.

Understand the beauties and downfalls of your technology. When does it fall down? What makes it fall down? Can you predict its flaws? How do you design for the vagaries of the technology you are using to make sure your audience have a seamless experience?

Design for coincidence: people will imagine a lot more into their experience than you have actually orchestrated. This is both an opportunity and could present potential problems.

For more see Jo’s papers on the subject:
Emergence Driven Research Methodology – A research methodology for evaluating location aware experiences
Design for coincidence: incorporating real world artifacts in location based games