From Interactive to Participatory

The emergence of immersive documents, wherein unreality is perceived as reality by the ‘reader’, is a consequence of three converging technologies:

  • networked/mobile computing becoming pervasive
  • multimedia becoming multisensory
  • interactive becoming participatory

Alongside this synthesis, we can identify five tangential areas of interest, in which developments contribute to the facilitation of immersive documents. These are:

  • enabling technologies such as virtual reality, made popular by devices such as Oculus Rift
  • developments in graphic art and design ( see Diagon Alley brought to life for the immersive Harry Potter theme park)
  • new understanding of creative writing techniques underpinned by research into transmedia, literary/narrative theory, scriptwriting and game design
  • the desire by people (players/audience/readers) to participate, evidenced by activities such as cosplay, interactive gaming, web 2.0, participatory theatre, films, e-books and exhibitions.
  • a small but growing interest from the library and information science community on the implications of ‘immersive’ documents for our profession (collecting, indexing, retrieving, preserving, making available to readers or users)

The usual definition of the term ‘reader’ is expanded here to encompass the person or persons experiencing, or participating in, the unreal, immersive document. This may be by engaging with a transmedia story, by joining the audience of an immersive play, or by interfacing with virtual reality technology to enter a virtual world. The activity of ‘reading’ thus becomes participatory, so that the reader perceives the documented world as a reality, and posseses the ability to make choices in the story, and influence the eventual outcome.

The image above shows the cast from immersive theatre play “Venice Preserv’d” in action; drawing the audience into a timeless world filled with contemporary meaning. There are an increasing number of participatory theatre experiences on offer to those willing to suspend reality and join the cast (if only at the superficial level of donning a robe and singing along) but the number of attendees at such events demonstrates the lure of participation.

In talking about participation we are obliged to mention the whole spectrum of video games – hardly a new phenomenon, but one which is becoming increasingly sophisticated and to some extent, perhaps merging with immersive, transmedia e-books, so that the boundary between what is a game, and what is participating in a story becomes blurred. The educational opportunity for immersive games is already evident, in prototype products including: In Ulysses: Proteus and Dolus: Finding the Journal of Odysseus.

On the boundary between what is a game and what is a participatory ‘story’, writer Mike Jones offers the clarification that

“A Game does not need, nor have to have, a Story.”

He uses the phrase ‘interactive narrative‘ to further distinguish between immersive games and immersive stories:

“Interactive Narrative.. a term which can encompass a broad range of experiences where the audience is asked to play a role, to participate or to engage directly with character and plot through action. An experience that involves game-play but does so in the context and service of telling a story.”

Old Dramatic Principles in New, Interactive Narratives. Mike Jones 11/08/2014. http://www.mikejones.tv/journal/2014/8/11/old-dramatic-principles-in-new-interactive-narratives.html [accessed 16/10/2014]

I think this is helpful in attempting to understand what sort of things could be immersive documents, and the differences between them.

Immersive documents do not yet exist. Today’s emergent versions are still reliant on the suspension of disbelief – but technological advances fuelled by the popular desire to participate are moving us towards documents that allow us to perceive an unreal story as reality.

Below is a list of resources supporting the move from interaction (where the computer generated world is separate from the user) to participation (where the experience is more believable). They emphasize the strong desire in many people to escape reality and engage with a scripted world. This listing is in its early stages, and it will develop over time.

Further proof: our lovely #citylis student @MeganWaples, participating in “Venice Preserv’d”.

Venice Preserv'd + Meghan

Immersive Plays/Theatre

The Kindness of Strangers

Thomas Otway’s Venice Preserv’d

Punchdrunk’s The Drowned Man – London

Punchdrunk’s Sleep No More – New York

Immersive Films

What it’s like to shoot a feature film for Oculus Rift

Immersive Books/Texts

The Craftsman

Immersive Games

Blood and Laurels

In Ulysses: Proteus

Dolus: Finding the Journal of Odysseus

Immersive Exhibitions

David Bowie is

Tomorrow

Diagon Alley Theme Park at the Wizarding World of Harry Potter

Cosplay

At Hyper Japan

Japanese ‘Cosplay’ craze becoming popular in London

Immersive/Transmedia Writing

Will virtual reality reshape documentary journalism?

The Writing Platform

Marie-Laure Ryan on Narrative Theory

Mike Jones: Old Dramatic Principles in New Interactive Narratives

Enabling Technologies

Julian McCrea from Portal Entertainment talks about how audience’s facial data can be used in immersive entertainment.

Stretchable electronics could lead to robotic skin, computerised clothes.

Unreality – the future of documents

At the start of a rainy February, it seems fitting to write about escaping into the unreal worlds of the mind, and the pleasing indulgence afforded by the inner flights of fancy conjured up by words on a page, a favourite memory, or simply the imagination. But what if we had some means by which these ‘unreal’ experiences could be perceived as real? A way in which we could experience fantasy worlds with the same sensory perception as that which we have when we engage with reality.

Developments in pervasive and multisensory computer technologies are leading us in just that direction, so that in the future, ‘reading’ a good book may deliver an entirely realistic experience, where the ‘reader’ participates in a simulated version of the story, and may be able to influence the final outcome.

These computer generated experiences could well be the next generation of documents, and library and information professionals should consider how they might fit within the information communication chain processes of creation, dissemination, management, indexing and use. It is entirely possible that we will see radical changes in information behaviour as documents become more immersive and pervasive.

Most of us enjoy becoming drawn in to a good story; the more vivid the text the more we enjoy the fantasy. I first came across the phrase ‘immersive text’ in relation to the Harry Potter series of books. Whether or not you are a fan, the popularity of the world of the school for witches and wizards drawn into the mind by JK Rowling’s words is undeniable. On watching the movies, I found myself wishing for a study like Dumbledore’s but I didn’t feel that I wanted to be any of the characters, and when I read about the Harry Potter Studio Tour, I didn’t feel compelled to visit a theatre set Diagon Alley. But it seems a lot of people do. Many fans want to drink in the world of Harry Potter, they want large doses of unreality.

But there is nothing really new here.  We are all ‘fans’ of something, and there are, of course, fictions, films, plays and games that tempt even the most dedicated of us all into unreal fantasy when we engage with them. One of the richest arenas for keen advocates of unrealism is provided by cult-tv and its close relation, cult-fiction. The exact characteristics which identify a television program as ‘cult’ are nebulous, but examples come readily to mind. Star Trek, as long ago as the late ‘60s early 70’s, spawned followers who afforded much time and effort in writing and distributing works of fiction related to the show (fanfiction). Unwilling to leave their engagement with the show alone until next week’s episode, they augmented their experience with fictions, poems, art work, songs and in recent years videos and conferences. As much as they could, they made (and still make) their imaginary world real.

Early cult-tv progams themselves encouraged the concept of unreal reality, by anticipating the technology which would make this happen. In the popular 1970’s children’s series TimeSlip, we see a ‘fantasy room’ which contained a tubular device (rather comical by today’s technological standards) which when placed on the user’s forehead allowed their dreams to be experienced as a reality. (Series 01, Episode 07 pt 2)

Wim Wenders’s film ‘Until the End of the World‘ (1991) depicted a more modern looking headset device, which again, allowed the wearer to experience dreams as a reality.

Whilst contemporary 3D films allow members of the audience to perceive objects as real visually by donning rather flimsy plastic glasses, a more convincing sensory experience remains elusive – within the realms of science fiction rather than science fact.

Ironically, it is science fiction cult-tv that shows us how this might work, and the holosuite – a concept popularised on Star Trek – The Next Generation is perhaps the most widely known portrayal of unreality tech. The holosuite is a space in which people engage within a computer generated unreality that is indistinguishable from reality. Fans of the show will be able to recall with ease, and in detail, all the best holosuite episodes but to summarise the holosuite could be used for shared experiences of entertainment (taking part in a crime novel or a going to a jazz bar), and also for examining historical incidents, or for training.

This unreality comes with issues however. Most centre around aspects of sensory stimulation – eating and drinking for example. Do you get drunk if you drink holosuite wine, and do you get fat if you eat all the computer generated cakes? But there are deeper issues: if you die in the unreality are you dead in the real world? If you fall in love is it ‘real’? More philosophically though, do characters generated in the holoworld have the right to existence? Once they have been created, do we have the right to turn them off by shutting down the program? One poignant episode considered whether a computer generated hologram had the right to leave the holosuite environment to persue his life elsewhere.

Fast forward to the 21st century and fandoms for newer television shows such as Buffy or BBC’s Sherlock enjoy even greater forays into unreality supported by developments in computing technology and social media. Crucially, the latter makes the distribution of fan-related works easy, and essentially, social media allows fans to find each other and to arrange shared activities such as cosplay, where fans dress -up to ‘become’ characters in their favourite unreality. Unreality is more ‘real’ if it is shared.

Cult-tv then, allows us to understand from one perspective, how far fans will go to make their unreality seem real.

But there is more. The ‘immersive’ adjective has moved beyond its association with traditional texts. Fuelled by the progress in networks and mobile computing platforms, electronic ‘immersive’ texts are emerging, which combine aspects of the traditional printed book, and televisual experiences. These documents reach beyond what most of us understand by the term e-book, in that the story follows the reader into the real world. Boundaries between reality and unreality become blurred.

In this new type of document, exemplified by ‘The Craftsman’ (Portal Entertainment), events unfold in real time, and engage their readers as part of the fiction. Readers receive texts, emails, calendar updates and ‘phone messages from other characters within the plot. The text plays out across a range of devices (transmedia), and can be put down and picked up again when convenient. Although we have the technology, the creative writing techniques to support this sort of fiction are in an early stage, and such immersive fictions are few in number. They are also expensive to produce, and reviews so far are mixed, with some commentators suggesting that action is limited and takes too long to update. These criticisms could be addressed in time, with more initiatives like “The Immersive Writing Lab”.

Yet the rise of ‘immersive’ doesn’t end with texts, electronic or otherwise. In recent months I have encountered ‘immersive-plays’ (Punchdrunk’s The Drowned Man), ‘immersive-exhibitions’ (David Bowie is) and ‘immersive-installations’ (Tomorrow )

The unique selling point of each of the above is participation. The viewer is invited to step into the unreality and live as part of the fiction; although each still requires suspension of disbelief on the part of the ‘reader’ or ‘participant’.

So at the time of writing then, none of this is really real. In spite of the popularity of engaging with highly elaborate fantasies, demonstrated by cult fandom, transmedia specialists, theatres and museums, there is room for improvement when it comes to delivering experiences which cannot be distinguished from the real thing.

The emergence of multisensory computing and network technologies does however, bring the promise of applications which offer us a more realistic fantasy than those which play out in our imaginations.

In order for unreality to work, we need technology that allows us to sense everything in the same way as we do if it is real. This goes beyond seeing and hearing, to include touch, taste and smell. Recent work on this type of multisensory communication so far leaves us with rather clunky, physical devices, which go only part way to evoking a sense of reality. Wearable tech is hardly a lightweight experience and even though some of the demonstrations help us engage with a plausible world, it is still impossible to forget the simulation whilst wearing a headset, gloves and other unappealing apparel.  (see this demo from UCL)

Participants in the holosuite are clearly perceiving the computer generated world as real, via all of their senses, by some, as yet,fictional neurological mechanism (photons?). Whilst we can speculate on how to stimulate areas of our brains without visible means of support, experimental work in this area flags up a few ethical issues to say the least. Nonetheless, work such as that of @AdrianCheok, shows us how much progress has been made in haptics, and the multisensory internet, and suggests that the rendering of unreality as reality may not be so much science fiction as we may think.

In his book, ‘Beyond the Library of the Future’ written in 1997, Bruce Schuman speculates on what will become of the library. He presents several scenarios for the near future, and in one of these he suggests that the library in the year 2022 will curate ‘experiences’ – rather than just physical works. The experiences are envisaged as computer programs which allow a ‘reader’ to engage with a recording of a real experience (memory), so that they perceive it as real for themselves.

If we allow ourselves to extrapolate beyond our current technological boundaries for a moment, there is no reason to suppose that these experiences could not be fictional, an extension of the ‘immersive texts’ suggested by ‘The Craftsman’, or indeed, entirely imaginary.

Whilst it is easy to comprehend the lure of fantasy, allowing us to enjoy something in unreality which we could never experience in real life, such immersive encounters could also support training and development in areas such as emergency response, surgery, piloting a plane or dealing with difficult customers.

Immersive experiences will undoubtedly have an effect on information behaviour. The use of Google Glass and smart watches are already instigating  questions of ‘information etiquette’. There is certainly an interesting future for library and information professions then, in trying to organise not just everything we know, but everything we can imagine. Every unreality.

My Absolutely Fabulous Life

Note: I refer to a Facebook profile in this post; I removed my Facebook account several years ago. 19/07/2016

——-

August. The month for writing. And my attention is caught by the recent PEW Internet Report which suggests that US millenials will continue to share information as they get older and take on more responsible roles. In other words – privacy is what people did in the past. ‘Sharing’ is the new black.

Hmmm – but what to share? Reality? The endless repetition of school, college, work, kids, cleaning and credit card bills?? Of course not. When did someone you follow last tweet about cleaning the oven? They didn’t. Social media requires us all to airbrush our lives into the kind of unreality mirrored on the covers of magazines – the ones where Madonna looks younger than her daughter and you are left feeling short and fat. Social media demands that we all use a special effects filter before uploading ourselves. We only tweet the nice things – specially chosen for their wit, charm, educational value and sheer brilliance. Are you sitting on the sofa watching a soap opera whilst eating mostly carbohydrate? No. Bet you are reading something on your college reading list, attending the ballet, practising the violin or spending quality time with the children – and for certain you are writing a book – an article at the very least – and that’s what it says on your Facebook status. It’s a bit exhausting – keeping up the charade. Hoping that no one you know ever sees the real you – the un-airbrushed horror. That, of course, is why I work as an academic from the seclusion of my attic. Ha! And at least having entered adulthood before computers were invented networked, (not a day over 28) I have escaped the need to drag around all the people who attended primary school with me (except Clare x). Nor do I have any idea what happened to those with whom I went to high school (except Sara and Karen xx). At least I have been able to move on – and quietly forget all those ‘friends’ who were never interested in me in the first place. But what of the millenials? Will it be easy to act as a managing director, when everyone on your friend list remembers you having a desperate crush on your French teacher when you were 11? Perhaps we will enter into a kind of bartering system, where no one mentions your bad calls, and you, in return, do not mention theirs. But …. oooh – the temptation. Remember all the times you thought you looked great/funky/clever/sexy/amusing? Well 30 years later those photos have a very different value. Only instead of fading in a box in the attic they are freely available on the net. And you don’t have to be a politician or a lecturer to squirm and flinch – anyone can experience the toe curling embarrassment at being reunited with their past. Pointed shoes/pink hair/rocky horror/snogging the guy who married your best friend  –  all give your followers cheap thrills. I have written previously on the niceties of ‘deleting’ (see Delete by Viktor Mayer Schonberger…) – where stuff you upload has an expiry date, after which it self-implodes. But the technology is not quite there yet. So if you upload it – it stays for good. For the amusement of all. Especially your children. And anyone who works for you.  And what of de-friending? Will it become as socially acceptable as de-cluttering your wardrobe? Carthartic perhaps? But can you ever be invisible to someone just by de-friending them? Nah … go Google …. or friend a mutual friend.

So what to do ? How much should we share? How much can we get away with?  (ok – 36 then ..)  and in any case what is everyone else ‘sharing’ about you? Worried who has access to your medical records? How can we stop them? Ever been ‘tagged’ in a photo you didn’t sanction? What about a video of all your lectures, even the ones you haven’t really delivered before and where you look like a troll? Especially those. In high definition.

Sigh. Can’t take on the entire world. Just have to hope that I haven’t said anything *really* bad or worn anything *really* un-photogenic. I am in fact, rather a non-entity when I Google myself, (yeah sad), but nonetheless, I have given some thought to my social media profile. As an academic, I think it is beneficial to project a warm, savvy persona – someone who has  insight into library and information science, and an interesting way of commenting on and interpreting the ideas of others in my field. I would hope to be convincing as someone you feel should be in charge of the class. So everything I tweet, blog or facebook does have a bit of a spin. I don’t mention bad hair days, or who I’m dating right now – I do try to mention anything relevant to LIS, and of course anything which will persuade you that I spend all my time reading, watching science programmes, attending lectures, exhibitions and art galleries, with just the right amount of cookery school classes, 80s pop concerts and walking on the beach in the rain – all to convince you that my life is really absolutely fabulous.

lynxi(ok – 49 then).