Posted on Fri 30 Jun 2017
A Tendency To Spill: the ups and downs of turning a short story into a chatbot.
Here I describe the benefits and drawbacks of choosing an interactive, digital platform as a means of increasing the reader's enjoyment of a story.
This project began with an 1100-word short story that I wrote a couple of years ago. I wanted to find out how much more of an intensely intimate reading experience I could create by using one-to-one digital interactivity.
For the funding bid, I decided the simplest and most effective way to increase the intimacy of a reading experience would be to retell the story as a chatbot. Whilst many good things have come out of it, I'm left wondering if what we've actually proved is that using a chat bot is neither simple nor effective. During the 6 months of the project I kept an ad hoc production diary, which I've found very useful as it's helped me to recognise my own process. Here is a write-up of the project: my intention with this is to show not only the benefits but also the drawbacks of choosing an interactive, digital platform as a means of increasing the reader's enjoyment. Following this post about the process, there'll be another one from me about the story world of A Tendency To Spill, so look out for that.
Start date January 9th 2017 I turn my third person prose into first person dialogue.
From the original story:
“Romy’s information about herself was that she was two years old. This was the date of her manufacture, the date she had been built by her parents Tom and Anne’s printer from the P13 design, the form of a thirteen year old human girl.”
As I translated it for the chatbot:
Again from the original story:
“At 4am, Romy’s internal clock woke her off schedule. An ‘unknown’ part of her system she had not encountered before had her sit upright. The lights in the room did not come on automatically as usual, she had to say the word ‘light’ and as she did so found her voice level was weak.”
Then as it became for the chatbot:
I change some major character traits from my original short story.
I have shifted the point at which we meet Romy, as the ‘robotic’ emotionally flat girl she is in the first paragraph of the original story:
“Romy casts her internal eye over the matrix. ‘Hmm. Now I don’t know what to do. This is unusual.” Romy is not used to not knowing what to do. There are rarely moments of indecision for her. She never questions herself. However, three times in the last three months she has spilled a liquid.”
To the insecure teenager we now find her to be in the chatbot version:
I turn the story from past to present tense.
Llive instant chat means the story is happening in real time as far as the reader/user is concerned. The original ending of the short story reads..
“In her room, Romy picked up her rucksack and packed it with a bottle of Vita, all her pairs of clean pants and her currency card. She didn’t see anything else to put in it, so she walked out of the front of the house and down the road.”
Whereas in the chatbot version we have..
Using a Google sheets format.
The spreadsheet allows both Constance and I to see the strands of text and coding needed for the chatbot.
Most interactive scripts I work on these days are done in spreadsheets. This is because there are several considerations going on simultaneously, one of which being what the audience/user is doing at any one time. So much of these scripts are based on ‘if this then that’ and spreadsheets are the most useful format. Constance and I created a project spreadsheet. After the writing I’ve done in film-script format (transcribing the third person short story into first person dialogue) I transcribe it to the spreadsheet and eventually I write directly using the spreadsheet itself. Constance creates a column for every possible word or phrase that the reader/user might input and this creates a good structure for me to create as many outputs from Romy as possible. It’s very helpful that Constance adds her own inputs as the reader/user. I respond as Romy and it gets the story work done a lot quicker. Eventually I write both inputs and outputs.
Using the online chatbot platform, Pandorabot Playground.
It’s new to both of us but it is easy to use for beginners and relatively cheap to maintain. Constance copies and pastes my words and her coding from the spreadsheet into the Pandorabot Playground. Constance writes all the coding and eventually I learn the basics myself so that I can make changes too. Eventually I end up writing in Playground rather than in the spreadsheet because you can test the chatbot, see the answers it’s giving and make changes much more quickly there.
From Pandorabot Playground Constance transfers everything to her Wordpress site, from which the chatbot is launched as a pop up window. The aim is that it looks like an illustrated book cover, followed by introductory text and the instant chat interface. The audience accesses the chatbot from the funder’s website, Writers’ Centre Norwich. On their webpage there is an introductory text to give the reader enough context for them to get the most out of the chatbot: the same amount of information that would be made available on a book jacket.
Helping the creative process.
One of the key advantages is the ability to strike up conversations between the developing character and a test audience member during the writing process. Using the chat bot to play question and answer, with me playing Romy and Constance being the reader/user, we’re able to develop the story world and explore any blank areas in the character. I will likely use this method again on other projects even if the story isn’t interactive, it’s a really good way of fleshing things out and filling in loop holes:
Tom explained to me “You’re not one of us. You are human. You grew up as a human girl but you don’t have that memory any more.”
Why don’t you have that memory anymore? Was there an accident or illness? Was it removed?
What that eventually looks like in code:
Whenever I write, the music I choose to listen to while I'm writing sets the tone of how I want the overall piece to feel. When I'm writing I prefer music without lyrics, as they’re distracting. Often I choose film soundtracks and this time I've chosen soundtracks from Blade Runner and Aliens. Although they’re pretty humourless and I'm not sure how humourless my writing will be (it turns out there’s not many laughs in this story.)
I want it to look and feel like a book.
Constance and I decided a few things about how the chatbot will be presented. We are going with the Pandorabot playground as I’d found it when I was looking for something to put into the funding application. Constance is asking Phil Hall (an AI expert) for help as this is the first time she’s tackled a chat-bot. We decided it would scroll down as one long piece rather than turning like pages of a book. We looked at more common-place chat-bots, which are presented like instant messaging. At the moment it will mostly be read/interacted with on mobile but also needs to work in a full screen browser.
I've made a pinterest board. Along with the music I listen to, this mood board allows me to set the tone. I've been looking at beautiful book covers as inspiration and I will commission Bart Blazejewski to create the illustration for me, based on my ideas. https://uk.pinterest.com/hazelgrian/a-tendency-to-spill/
I've engaged a graphic designer, Bart Blazejewski, to create ‘front and back cover’ illustrations. I want the project to feel like a book as much as possible. The idea is to create intimacy so that the reader feels comfortable enough to feel the intimacy is appropriate and a satisfying experience in the way a good read would be.
Brainstorming and shuffling ideas.
I am writing my thoughts, ideas, phrases, words onto post-it notes and mixing them up to shake up the writing process instead of getting stuck in a rut and writing in a linear way. I literally shuffle the post-it notes and then transcribe them as a list in that order. Some of these random juxtapositions tie themselves together neatly and show me the themes I am bringing up with my ideas. This has worked very well and I will use this ‘shuffling’ as part of the writing process again.
So I'm still working out the timeline - time passing and order of play for the story. Does a greater amount of time pass between chapters? There’s more artifice in chapters with time lapse but that might be a good thing making it feel more literary and dramatic given its context i.e. a box on a web page.
“It didn’t recognise hardly anything I asked it!”
So now Constance and I are ready to test and already we’re finding there aren’t enough possible inputs in terms of what inputs the reader could possibly put in - I had a fairly quick test the last two days and it was very frustrating because it only recognised the particular phrases we had put in and they were far too few.
March 14th 2017 Big crisis after feedback round one: It needs more work than I realised.
It was a shock to get negative technical feedback from the funder with no reference to my writing (“It keeps spewing random nonsense at me.”) It was upsetting and I've got a part time job to do so I can't immerse myself in it until it’s done. I feel like crying. I realised I wanted the piece to showcase my writing in a way that has been missed before. Previously the interactivity, the platform and the things the reader/user has to do in order to discover the text has meant a lot of my storytelling has been missed altogether. What is perhaps more important is that a critical review of my writing hasn’t taken place because of its nature: experimentation with digital platforms and online audiences rather than the pursuance of excellence in an accepted craft. I wanted this to be a better opportunity to be read and appreciated by an audience but I have to sacrifice that for this chatbot experience. I feel we have some way to go before i feel confident that it is interesting and moving as an interaction. To prevent the chatbot from being very frustrating, we are having to rethink the essence of a lot of it and the plot devices.
My best response is to get help immediately: feedback round two.
I asked Rik Lander to advise, particularly because of his experience in telling interactive stories using AI. Rik gave his feedback on some pretty basic building blocks of my narrative and experience as it currently stands. At this stage, as I presented it to him, the reader/user discovers part way through the narrative that they are infact role playing as Romy’s sister. Rik’s feedback was this:
The call to action isn’t clear. He says he didn't know what to say. He didn’t have enough information and didn’t know what he was supposed to be doing. How does Romy need help? These things need to be made clearer at the beginning. Rik helped me clarify the language of the call to action, making it much clearer what the reader/user is being asked to do. His advice actually lead me to change some of the basic story elements because I needed to take away a lot of the deliberate obfuscation of the story in order to guide the reader/user through the unfamiliar interactivity. The relationship between the reader/user and the character are symbiotic.
The chat-bot’s answers are too closed off. They don’t invite the reader/user to keep chatting. This has come about because I took the decision to avoid using any questions in Romy’s responses in order to avoid exposing a weakness of the chatbot that means it’s unable to respond to any specific answers given by the reader ie it can’t follow up and carry on that strand of conversation. Romy has to steer the reader/user in a new direction all the time. However, I had to then decide to start using questions in order to open up the conversation, which has lead to the problem as predicted, that Romy can’t really converse very well. Constance and I had to make the decision early on that Romy doesn’t ask questions of the reader. If she did then the readers’ answer would warrant another reply from Romy that makes sense. We can’t build logical conversations, so Romy only makes statements or gives ambiguous answers as well as specific ones. Rik Lander gave some tips on the kind of things Pandorabot can do and on predicting inputs and creating authentic outputs. Later I had to abandon this approach in response to Rik Lander’s feedback about everything Romy said being a dead end.
As a result I had to rewrite everything Romy says: in order to make each one of her outputs more inviting to the reader, to illicit a response from the reader and to keep the conversation going. I may have gone too far the other way as it was later fed back to me that she always asks a question but isn’t able to keep the conversation going. Exactly the reason why she asked no questions to begin with.
I have to decide if it’s an interactive story or instant chat. You can't have both. On the issue of the larger paragraphs of linear narrative interspersed with shorter chat, Rik says that these don’t sit well together (we had this criticism from another tester too). An 'interactive story' would mean mostly reading with the occasional very simple piece of interaction such as selecting the next chapter or looking at additional information. In this project that's not what I want.
I want this project to be about conversation. In this case instant chat, so I’ve had to make the decision to break up the larger paragraphs into answers from Romy. Those longer paragraphs giving the key points of the story in a linear form were a chance for my prose to shine. However I’m disseminating them into the chat script by breaking them up into small points.
Test - On Rik’s suggestion I’ll pick out 20 key points that will remain the backbone of the narrative and we will see if we can find a piece of code that allows us to deliver them in narrative order. I’m trying to make sure there is a linear backbone adhered to as well as the feeling that the chat is natural and isn’t preprogrammed. Ideally the nonlinear chat gives the reader a sense of agency while those key linear narrative points deliver the story.
Result - As it turns out, due to limitations of the platform and/or our coding knowledge, we aren’t able to have anything appear in a particular order. I feel that the final version of the chat is circular rather than linear. However Constance feels that’s not quite the right description: for herrather than ‘circular’, it is ‘diffuse’ instead of linear. It is still luck of the draw as to whether the random comments make sense to the reader/user and if they have another go then they will see that.
Role playing is confusing. Especially if you’re not given enough information to input satisfactorily. I agree and have now made changes, getting rid of any mention of the sister and placing the reader/user in the position of themselves, giving Romy advice on whether to stay or run away.
Give the reader key words and prompts on the chat page so you can easily scroll back and refer to them e.g. print, original, The Wall, Tom and Anne.
April 20th 2017 We may have to abandon one of my central creative points:
Test - The delivery of key story points in a linear narrative order.
Result - This turned out not to be possible during this project either because Pandorabot isn’t capable of it or because we couldn’t work out how to do it, even with help.
Test - How do we not reveal plot points ahead of time?
Result - There’s no way of controlling that with this system. It’s a random delivery of the story, all depending on the direction of the conversation taken by the reader/user.
Test - Does having no linear narrative element mean that I am sacrificing the satisfying notion of an ‘ending’?
Result - I feel it probably is. Without it the end is simply when the reader has had enough of chatting.
Coding is part of the creative process
I think it would help if I really was able to learn the rules of syntax so that I can use * ^ without the coder having to do it. So I can express myself that way too. Eventually I learnt this and was able to make the changes I needed, it helped with the creative process.
For Constance’s part, in retrospect she feels she did not give enough input as to content and concentrated on the coding side of things. She feels that had she been more forceful re creative text ie being non linear from the start, and creating more vague responses that worked with different user input, then possibly the whole process would have been quicker. On the other hand she believes that I, as the author needed to arrive at that conclusion myself after experimentation.
Invite more testing and feedback.
Contrary to my own advice to other people, I haven’t invited much early testing and feedback. However we have included a feedback form with the chatbot. Whether we will be able to make any changes in response to that feedback is another issue, that sits alongside keeping up the Pandorabot monthly payments. Again contrary to my advice to others, I haven't made a long term upkeep plan.
Feedback round three.
I have asked Dr Tim Kindberg and Dr Tom Abba to give it a test, as they’re experts in AI and interactive narrative respectively:
- Unintentional repetition of (the random set of) replies ‘takes you out of the experience’ (I agree) and he felt that he got ‘no plausible responses’ (which is disappointing). When I gave Tim the background he was missing, he concluded that the story has been lost. Tom was a lot more positive: Whilst he had the same complaints as Tim he felt he got more out of the writing and ‘intrigue’ of the story. Having said that, he was unaware of most of the story. I realised I should have given them the copy that I’ve written for the website, which puts the chat into context.
- Would I rather just read the story and not bother with interactivity? Tom felt that the unintentional repetition and lack of in-depth conversation lead him to wonder if perhaps he’d rather simply read the story and not have to bother with the interactive interface.
- You can never have enough random outputs. These need to feel relevant and part of an authentic conversation.
- To answer the key problem of the bot repeating itself, which is very off-putting, I ended up adding random outputs where by Romy actually makes reference to her repeating herself and how she is not always able to remember what she's already said.
- Repetition is the hardest thing to eradicate and the one thing sure to spoil everything. Having her mentioning it herself was my way of persuading the reader to be more forgiving of this fault. We don’t know how long they can chat before everything feels repetitive i.e. until the answers start repeating.
- It really needs the introductory text to explain what you’re supposed to be doing and for how long. I’d discovered that ages ago when Rik tried it but somehow I’d lost track and some of that clear information was missing. Tom had assigned himself a role anyway, as a kind of therapist. The key element as Tom pointed out was to put back the call to action and how long to spend on the experience. So I’ve once again included the suggestion of 10-20 minutes. Although in talking with Tom I realised you probably need about 40 minutes to an hour to gather all the story points. Tom also hadn’t noticed the illustration, that it acts as cover art and has an explanatory strap line. Tom helped me clarify the information given on the chatbot page itself and I’m now satisfied that the story context and the call to action are in place.
June 6th 2017 Final delivery and Conclusion.
The URL, website copy and image finally delivered to the funder and they have put it on their website! It’s been a long slog. It’s a part of my process to write an evaluation as I go and I have become more convinced that the best thing to do is to provide a criticism of the experiment alongside the chat-bot. By this I mean a blog post or series of blog posts discussing the pros and cons of using a chatbot to tell a story. I might record a podcast where I am in conversation with Tim on the issues surrounding telling stories using AI. Perhaps it would be interesting to have Tom Abba involved in a similar recorded discussion. Tim suggests that it would be useful to ask this big question:
What’s a ‘good’ experience of story?
I’m left questioning whether publishing my writing through the medium of interactive digital platforms such as a chatbot serves any useful purpose other than as an experiment with a very limited audience. Using these interactive platforms whether multiple or single doesn’t seem to serve me well as a writer. What's needed:
- A culture of criticism of the writing itself, which is not masked by the attention given to experimentation with platform.
The ability to attract a sustainable audience. Experimenting with platforms doesn’t give anything particularly useful. The way I use emerging digital platforms they are either unstable or not associated with creative writing. Each time feels like throwing something into a pond that creates no ripples. Nothing remains of stories because projects aren’t properly documented and archived. Digital writing is tucked away in the back of a digital drawer, often lost permanently when domain fees aren’t kept up indefinitely.
- Stable digital platforms that are associated with creative writing.
- Proper documentation and archiving of digital creative writing.
A Tendency To Spill as a chatbot is not entirely satisfactory but I don’t regret creating it. The process has allowed me to develop a story in a way I wouldn’t have if I hadn’t been able to respond to testing the chat bot with a Constance in the early stages. It’s actually an interestingly collaborative part of the writing process that I might use again - it might be useful in the development of another story. Working with Dr Liz Evans at Nottingham University has also given me food for thought. Her research has revealed what audiences value most in stories (in any medium) is not the same as what interactive writers look for:
Being captivated is the most desirable state for audiences: that feeling of not wanting to leave the story, of ‘bingeing’ on it.
Writers of interactive stories say that ‘conversation’ is the most desirable result for them - but for audiences it is not.
- What makes a story captivating is its character rather than whether or not it is technically autonomous.
This is something I will bear in mind for my next piece of work. What I will do with A Tendency To Spill is to take the story that has been developed by this chatbot project, and translate that writing back onto the page as a more developed short story.
Posted on Fri 30 Jun 2017
Original sin: Science Fiction tropes are up-ended in this Margaret Atwood-like Young Adult story, inventively told through instant messaging.