VR Environments

The rise of Virtual Reality (VR), Mixed-Reality, and 360-degree storytelling in art and performance contexts necessitates new embodied “languages” as audiences/users are challenged to navigate and become immersed in mediated worlds. Commercial content-creation of immersive, interactive games narrative often rely on tropes of violence and shooting. But as visual artists and theatre practitioners explore these mediums, they are imagining and producing VR environments that offer alternate modes within these platforms, what I am calling “strategies of resistance” to more commercial exploits. I have spent the last five years trying out as much VR as I could, from pruning Bonsai trees to a Squid Game VR game, to a Tempest in VR, and much more. In the examples I highlight here, largely from the initial period of investigation, audiences/users transition from shooting and reacting with violence to having tactical engagements that show an embodied perspective for the form’s development, expansion, and resistance through art and performance.Early adopters and an industry keen to capitalize on emerging and developing technologies have been quick to find and develop content for VR experiences, which, from my own participation at the South by Southwest Festival in 2018, ranged from entertainment (as you’d expect—gaming, movie add-ons, 360 film, sensory exper­iences) to medical innovations, real estate viewings, deep sea diving, or exploring fossils in the Museum of Natural History. Some key questions I had were: What are the experiences and environments that work well in this format? How can form and content merge to tell stories that are captivating and relevant? How can these technologies help performance and artists to tell stories differently?My interest in evolving media forms and their relationship to performance has always driven my research and has recently been fueled by my involvement the UK’s Arts and Humanities Research Council grant at Royal Holloway University called “StoryFutures,” a center for digital and immersive storytelling. Current tech research, marketing, and industry consultation demonstrates a focus on Virtual Reality, Augmented Reality, Mixed-Reality, Immersive 360, now being driven by innovations in AI. For each of these platforms, “digital storytelling” is a key to their success. But what are the languages this storytelling requires? How do these technologies challenge more “traditional” performance experiences? The opportunity of this grant was to include longer-term, in-depth academic research in creative industry projects, but it also presented possibilities for bringing industry/technological innovation into performance. Because these new forms so often become co-opted platforms for companies to brand and market commercial products or be taken over by games companies, it is easy to overlook the creative interventions by artists. Much like the introduction of the portable video camera influenced artists like Nam June Paik, Carolee Schneemann, Meredith Monk, Joan Jonas, John Jesurun, The Wooster Group, Cathy Weis, and others, more accessible VR systems are being used as new tools for new forms of performance and art-making.Content, whether explored through drama, performance, art, or entertainment, has to help shape the experience, especially as it relates to audience/user bodies. There is much work being done about VR and other technologies, such as the play Ugly Lies the Bone by Lindsey Ferrentino, which I saw in 2007 at the National Theatre in London. In it, a soldier returns from Afghanistan and works through her PTSD using VR therapy (simulated through projections on stage). Drawing from research showing VR’s therapeutic possibilities, the play represents the VR application but does not allow audiences into the experience. When it comes to actually putting audiences into VR, I suggest that, for the most part, form and content have not yet matched up consistently, which could very well result in VR as another gimmick of multimedia performance experimentation. There are exceptions and as the VR glasses and other immersive/interactive technologies become more portable and accessible, they will benefit from artistic experimentation.As I travel through VR experiences, I have been concerned with how markers of a corporeal experience do or do not play out in experiences like these, and curious about what might be shifted to produce resistance within these experiences. Those discussed here work through how they encourage very different types of relationships: from active agent to passive receiver of information, from agent observer to attempts at disembodiment or creative expansion of embodiments. Many of these modes overlap, but in each case VR challenges our corporeal perceptions in ways that create new possible languages for future development of these forms in performance and art practices.Active shooter: this term is too well-known describing (according to FBI.gov) the person or people involved in gun-related violence in populated areas. Within the gaming world the term for these fictional roles is “first-person shooter” (FPS), and there are many FPS games that revolve around the main character—the person playing—carrying and using a weapon. While this position is familiar to many active gamers, I learned it firsthand when I went to “The Void,” a VR Star Wars experience in London. In it, I became an active, first-person shooter in this fictional world without even realizing it. The nature of the hand-held devices, from gaming to VR, has facilitated shooting action, and after this experience I found myself wondering how else this “trigger-finger” could be employed in performance. Wearing sensor backpacks, vests, and headsets we were handed our weapons: Stormtrooper blasters. Given our “mission to “travel to the molten planet of Mustafar . . . to recover Imperial intelligence vital to the rebellion’s survival,” we entered the playing space. With headsets on, we saw each other as disguised Storm Troopers, individually identified only by our heights and a distinguishing color. In VR we traveled in a spaceship to a world to face the evil Storm Troopers and emerged onto a ledge in this action-packed apocalyptic environment.We huddled waiting to cross a flimsy bridge over a lava pit to the other side of a mountain. The description cannot convey the building adrenaline in our bodies, a sense of fear, and the unknown that facilitated what was to come. VR creates a realistic sensorial experience where height, depth, and a complete sense of being within this experience reorients your brain to “be” in simulated worlds. As I tried not to fall off this ledge, the “enemy” appeared and started shooting at us, registering physically as pulses on the sensory vests. Simultaneously, huge creatures appeared around the lava pits and began attacking. We saw shots coming toward us and without a moment’s hesitation I aimed my gun and began shooting anything in my sight. As I shot, finger in constant action, my sensor vest was reminding me with many electrical zaps that I was being shot at the same time. These little zaps made me shoot faster and faster, more and more . . . Stop. What was I doing? I am anti-gun, pro-animal life, and here I was gleefully and full of adrenaline pumping round after round into the “enemy,” and into these creatures (I remember distinctly aiming for the open mouth of a large beast coming at me). I finally got across the bridge and was shuttled into safety, and I took a moment to remove the headset. Actually, I was surrounded by empty gray spaces compartmentalized by temporary walls.In this expensive twenty-minute experience, I was changed somehow. I was unprepared for how this action—in the moment sensorially experiencing, but free of anything reminding me of myself—facilitated my ready uptake of this weapon, and how easy it was to just point and shoot without thinking. What was this experience intended to do? I was active, but did I have agency? Following this troubling (but fun) experience I set out to find alternatives—to violent embodiment, to shooting, and to interactivity within a VR space.Theatre and performance offer several examples that position the audience/user as observer, triggering a live action in specific moments, but often without influencing the narrative. To my mind, they have not yet merged form and content within the storytelling to be fully effective. Two works, Frogman (2017) and Somnai (2018) use specific types of experiences that work successfully in VR to build segments of the narrative. They exemplify relatively nascent stages in this evolving form, and as I write, I am also aware there will be more sophisticated developments already in production. However, they signal a shift from observation to interactivity as artists begin to hybridize live and VR performance.In Curious Directive’s Frogman, a fairly conventional theatre piece in which audiences remain in a “passive” position sitting on swivel chairs surrounding a long-playing space full of sand, watching a story about the case of a missing girl, Ashleigh, as told through the memories of Meera, a woman being questioned in a trial case. The VR in the piece alternates between scenes in the past—in Meera’s bedroom with her friends around the time Ashleigh disappeared, and also underwater sequences of Meera’s dad, a diver looking for Ashleigh’s backpack many years later. Audiences are instructed when to put on the headsets and alternate between scenes in the child’s bedroom, and underwater, diving with the father. In these scenes, all filmed in 360, the VR was dramaturgically integrated into the story so we could see the memories and watch the “tapes” of the dive as evidence, though it was never really clear who we were. (At times we were supposedly the jury, but nothing came of that).The VR segments gave us clues to the back story, to the days leading up to the disappearance, to the feelings and conversations between the children, but we were not active participants in these segments, just observers. In the diving segments, we were watching the evidence in the script, and the VR allowed us to feel involved as part of the diving team. When integrated into the storyline—as members of a jury watching evidence—the VR works well as a storytelling device. VR can feel magical, and because the VR headsets simulate the dive masks it is a good match. Integrating VR into a hybrid theatrical experience offers perhaps more opportunities for building new forms of storytelling, though in this and the next example, audiences were only placed in an observational position. No hand controls were options in these pieces, only visual experiences.Billed as “the place where hyper-reality and virtual reality meet,” part immersive theatre a la Punchdrunk and part VR, Somnai was a mixed-reality experience ranging from face-to-face theatre to site-specific exploration, from projected environments and VR to an escape room style challenge. The theme was “lucid dreaming,” and participants were placed in a familiar immersive theatre audience configuration, moving as a group and being asked to do tasks (put on a gown, have your face scanned). Led by a guide, we alternated between more traditional theatre “scenes”: in one we were questioned about our dreams, and in another we watched projections of stars, to scenes in VR. In one room we donned VR headsets on a swinging seat and “flew” through the sky alongside some CGI rendered geese. In another, there was a magical mushroom-plant forest where you could physically touch the plants, and through yet another, a futuristic desert. These spaces showed off what VR environments could do to place you into spaces you might otherwise only imagine.As a proof of concept for the company Layered Reality, Somnai shows an emerging form that blends theatre and media to offer an “experience” somewhere between the two that is not yet fully hybrid and does not fully cohere as integrated. The VR experiences within the narrative were short and broken up into three distinct moments around themes of sleep and dreaming and death, but it felt muddied and not entirely intertwined with the VR segments. The piece ended when we were taken into a room of beds where we put on our VR “sleep masks” and headed toward a white light, but it was not dramaturgically clear. Somnai did produce that familiar rush of adrenaline in VR, going through portals into different environments and trusting sensations of stepping into a void or flying. VR tricks your brain. Here we were more active than in Frogman, but also being ushered from moment to moment, restricted by the timing of the groups going through. I would call us agent observers—within the constructed environments we had some agency (to answer questions or carry a prop or move within the various VR environments) but we were also observers in this mixed immersive-VR experience, watching actors or being told what to do by others.Reviews for these more theatrical shows confirm my experiences—that the weakness is in the storytelling. Though I would add that the lack of bodily agency disallows the narrative segments to intertwine for the spectator. Who we are was not as clearly defined as the Star Wars experience, and then we were clearly stripped of our own markers of identity. In these diverse examples the fully immersed Star Wars was the most comprehensive, perhaps because we had a clear directive and some agency through the controls. In all of these we as experiencers could not follow our own path or explore the world for long, as audiences can in the now familiar immersive theatre of, for example, Punchdrunk. As the VR forms are more developed, story has become central to the experience, but the lack of agency remains.It is perhaps telling that the more recent Layered Reality projects, The Gunpowder Plot at the Tower of London, and Jeff Wayne’s musical version of The War of the Worlds, are both labeled as “Immersive Experiences.” These two pieces are more tourist attraction than theatre. The formula has been concretized following Somnai so that small groups are taken through these stories by actors, engage in some suspense or “scare experience” moments and interactivity, and have three moments of VR—in these two there were boat rides and some form of flying as well as one observation of a key plot point—but in none of them do audiences have hand-held control or any agency. The VR is well-executed and now fully integrated into the narrative, transporting your body in different modes to follow the plotline and each with a different “wow factor,” but it feels more akin to a theme park ride than theatre.As VR/360 video, immersive and mixed-reality tools become integrated into performance (already seen through Punchdrunk’s successes, the recent Builder’s Association’s augmented reality production Elements of Oz [2015–2019], and the many examples of immersive, games-oriented, experiential drama over recent years), and theatre-makers and scholars will have to calculate how a shift in audience/participant/experiencer agency and embodied experience can influence the storyline while challenging the viewing experience. In the examples above, I would have liked to follow my own path or have an option to not shoot back; I would have liked to understand my “character” in a material sense. In Somnai, I wanted to interact more with the VR environments, to see myself as one of the geese or explore different pathways understanding why we were there. VR is frustrating when it replicates the conditioned or the of or is to these digital languages and experiences we through how the form, and work VR has theatre and visual creative new to these forms and their artistic is to allow the audiences and to with the hand-held controls and find new modes of with experiences. VR experience such as and to use the controls to through the moments and each a to through the or around you as and of a all while in your piece is both and very physically on your to the controls to their to and in this space. that whether a piece of or a or about one about about your and here I felt more active than in other VR be that the languages of the games controls and hand-held are where the of experience, and audience can be most in the development of this The of from and to touch and controls has been in most performance but of resistance when taken in the sense of within VR performance some see my in in to and control when placed in VR performance, these controls on new audience experiences with performance this bodily agency within VR, I in which from to in and In this VR audiences from around the world in with their own VR as an audience alongside an who was a version of The Audiences up for a and entered exploring the large space on their out what the controls might you in any for up with the Audiences saw each other as with masks and were with playing many of the We could see our as we on to and tried out the controls. experience came to an active, embodied experience that narrative with gaming while to see what they might do within a VR space. the to through space in the use of possibilities for that me across and even into explored in other VR art experiences, such as of VR at the in in which you could into the within we the and a version of we also could explore and interact with the space and others, producing a sense of and the to the to In many VR shooting is not in to this it is used to in the or on the or used to up and move or move you across the space. But shooting is a use for the control and some research that also a The are However, and futuristic possibilities are shape new possibilities that remove from the With the of hand-held controls within performance and art audiences are new languages of embodiment, and The challenge of the in this moment of such being done by violence in the but with the controls can be was in this example, a VR art experience, by visual where the possibilities of new languages fully took shape in my I it in at an at the called The was three of various VR and augmented art works in a which of time and two people at a time could do this The material that it by the of the through the magical of the is of of her body in the In the experience, the while in VR were projected so people could watch as the Given hand-held with the on you could point and a of and full of As you the came into focus as perhaps of and from the from other from a allowed me to explore this from above, where a to within the In this world the the same that gave me finally as an to the Star Wars experience. I the and the and to my a hand I and in and after of many different I the the in any and would to I the could from other and I at the same and got an of two I to but there was a waiting to add to the of I as I felt at in what could have been an apocalyptic I was to create a new world around me without a world with possibilities of and any In this VR and I many more to I finally found new languages within which to commercial and the of artists and perhaps the can be put to good