Posted on Wed 12 May 2010
Communicating Emotion: an unscientific encounter with technology.
It’s become popular to use technology to ‘measure’ emotions as part of an interactive 'process'. But do we really want to know how fast our heart’s beating or how sweaty our hands are? And if a computer knows this how can it really translate these things into anything useful for us…

Posted by

Hazel Grian
A pioneer of interactive drama, specialising in robots, Hazel began as a live performer & filmmaker. In a 35 year career, she is driven by her ambition to bring stories to audiences by ‘any medium necessary’.It’s become popular to use technology to ‘measure’ emotions as part of an interactive 'process'. But do we really want to know how fast our heart’s beating or how sweaty our hands are? And if a computer knows this how can it really translate these things into anything useful for us emotionally? The scientific measurements of our physiological responses, which are out of our conscious control, are not necessarily the most useful way for us to understand emotion through experience; they are a scientist’s reaction to emotional understanding. For me it’s more important to ask how do we show that we’ve understood these emotional signals and how do we respond? In other words how do we have an emotional conversation? How do we have an empathetic exchange between human and computer?
I want my audience to consciously make decisions about the emotional state of the robot and about their own emotional state, I don’t want the experience to feel like some kind of lie detector test. My aims are about decisions and control in the hands of the user, not about scientific testing. We read other people’s body language to seeif it’s safe for us to approach them and if we’ve been understood by them when we do interact. These things are important and we judge consciously and unconsciously facial expression, heart rate, breathing, blood pressure, sweating, muscular tightness and posture but what we’re looking for is distress or pleasure (apparently we have the ability to express and detect distress and pleasure from birth) but what makes it actually meaningful is a sense of empathy. Verbal language isn’t essential to convey emotions, a dog or a baby will know what emotion is in your voice no matter what words you use. So you can say and read a lot through sensory expression alone, voice intonation, facial expression and tiny bodily movements. I don’t think we’d choose to communicate with another being through knowledge of their heart rate or sweat levels; if we had to find a common language we’d much rather communicate by sharing something abstract like music.
Emotion as a word is imprecise and imprecision can be useful. Creating a mood may be the best way to get results. Mood bias makes people interpret ambiguity differently and tells us more about them, it gives people more space for expression if you present them with important ambiguity within a ‘safe setting’. Meaningful interactions between people involve warm hands, eye contact, movement towards or away from each other, smiles, head tilts, listening, breathing, hands and feet movement, tears, head up, listening, reacting. Speaking, responding, having a conversation, moving closer, touching hands,responding with noises, making jokes, sharing secrets, saying nothing, sharing moments.
It is not the realistic humanoid form that is important, it is the way it responds and interacts with you that is important: giving it an intelligent living entity feel that has emotions and empathy and needs your help too. We may as well let the robot be and do what robots do best and not try to make them look or move like humans. Robot mentality may be different to humans it should be an interesting philosophical engagement between us and them.
So what’s the best way to convey contact and understanding between a computer and you? Does it read your brain waves and speak back to you in headphones? Using BCI translated to audio recordings? I have been thinking I could try to make an engagement like Close Encounters – using music and hand signals to communicate. Or like The Abyss. Imagine the next scenes after these films have finished, how do you begin to talk and communicate? I’ve been listening to the band Sigur Ros recently as I just discovered them. Their album Agaetis Byrjunis very emotionally evocative. Their use of an invented language creates thekind of ambiguity I have just mentioned, allowing the listener to interpret the meaning in their own way, lead by the emotional language of the music. The changes between the tracks create a kind of narrative, whatever order we listen to them. I am wondering, despite not having the rights to the music, if I could use this album as means of creating mood during which we interact with the alien-like entity that is theDaemon. I am thinking that the Daemon experience, whether for individuals, couples or groups, is a way of addressing loneliness and depression and is a kind of therapeutic experience, whichwordlessly brings hidden things to the fore and deals with them in a universally meaningful way.