Mixed reality artists and ACMI X residents Emma Roberts and Ben Andrews explain how they're pushing the boundaries of storytelling.
"Open your eyes, listen, smell and feel – sense the world in all its magnificent colours, depths, sounds, odours and textures – this is the cinema of the future!"
In his 1955 paper titled ‘Cinema of the Future’, Morton Heilig articulates a radical vision where stereoscopic images and spherical audio (two hallmarks of contemporary virtual reality) mediate states of total sensory embodiment. By immersing our perception within a seamless, motion-responsive virtual world, Heilig argued that the next step was to manipulate the remaining senses, treating the nervous system like a musical instrument and completely reproducing the way we experience reality. When these conditions are possible, the theory contends, the paradigm of “visual art” would eventually shift towards a new experiential framework: an “art of consciousness”.
In order for a VR experience to transport us to boundless three-dimensional worlds, we need to be completely immersed in sound and vision. This effectively separates our primary awareness systems from our body. Using VR inside a living room, this splitting between two worlds might hardly be noticeable (beyond the acute paranoia of being watched). Yet this bizarre crisis opens an important and incredibly fascinating space. As with any sensory isolation, the disjunct between vision and the other faculties of feeling gives rise to profound new possibilities for sensation. STARLESS is all about embracing this uncertain sensory architecture.
Within the spectrum of mixed reality, there’s a saying that ‘1+1=3’, meaning the response is often more than the sum of its parts. Multi-sensory elements like wind, heat, scent and haptics, as well as exhibition design, enhance the virtual experience and intensify the sensation of being present within another world. We extensively research each stimulus according to different forms of audio-visual stimulation, assessing both its unique properties and combined affective result - an alchemy which invariably staggers and bewilders us. This experiential approach to VR design directly addresses the role of the sensing body, dissolving the boundaries between interior and exterior and reconfigures the virtual experience in pursuit of inducing altered states of consciousness.
Our work in multi-sensory VR – often termed “mixed reality” – explores this very space. We create immersive installations which combine VR and environmental design to guide audiences through embodied narratives and transformational experiences. In 2016, we produced two site-specific multi-sensory works: The Moon Is Gone & All The Kings Are Dead and allthestarstheybleedtogether, both of which have fed into the upcoming STARLESS for Melbourne Music Week in mid-November.
Moving beyond the limits of hearing and seeing, STARLESS invites audiences on a sensory journey outside of their bodies. Encompassing scent, heat, wind and uncompromising quadraphonic live sound, the 15 minute multi-modal installation is one part immersive theatre, one part live performance. STARLESS recasts the VR headset as an audio-visual blindfold, blending physical and virtual spaces, while bringing participants into an altered state of consciousness.
Internationally, there is a growing breadth of VR work adopting multi-modal synaesthetic design as a means to maximise the immersive encounter: Alejandro Gonzalez Iñáritu’s extraordinary Carne y Arena; Doom Room; Tree, and the work of Jane Gauntlett. Almost every major festival for VR in 2017 has included at least one multi-sensory piece, speaking to the growing demand and excitement for the medium. And yet mixed reality is yet to break through in Australia, leaving an exciting space to introduce local audiences to new modes of experiencing the virtual realm.
The impulse towards multi-sensory VR in these experiences is to maintain a continuous audio-visual environment. For example, wind in the exhibition space corresponds to a helicopter flying overhead; haptic feedback gives tactile, bodily emphasis to significant moments. It is in this world that STARLESS seeks to intervene. In partnership with Melbourne-based VR studio Liminal VR, this project explores a range of phenomena associated with disembodied experience. It incorporates neuropsychological research into the triggering of cross-modal sensations, tricking the nervous system into producing a range of aberrant and hallucinatory responses. By modulating sensory information inside VR, this work seeks to recalibrate the habitual patterns that construct our everyday reality inside a narrative framework.
VR, as Heilig predicted many decades ago, is barreling down a path towards producing completely unified sensory-based experiences. As the moving image shifts from the frame of traditional cinema to the interactive framelessness of VR, so too is the phenomenology of spectatorship, narrative and the environments required to activate it. VR is not an empathy machine. VR is an experience machine. And the optimal design for VR must necessarily incorporate comprehensive consideration of the various senses, their inherent connectivity and the exact intensities and rhythms needed to transcend those individual borders. Here, the transformational, epiphanic or sublime may become accessible, even desirable, narrative states. This is what we see as the native language of multi-sensory VR: a language of sensation.
STARLESS was presented from 17-21 November 2017 at the The Facility’s Dark Studio (Rear 2 Chelmsford Street, Kensington).