Ways of (not) seeing: Structures of Visibility in VR

A Brewster pattern hand-held stereoscope, Victorian period, fixed focus prismatic lenses, mirror, glass, screen, maple veneer. Photo courtesy National Science and Media Museum/Science and Society Picture Library, UK
A Brewster pattern hand-held stereoscope, Victorian period, fixed focus prismatic lenses, mirror, glass, screen, maple veneer. Photo courtesy National Science and Media Museum/Science and Society Picture Library, UK

John Berger’s Ways of Seeing provided a 1970s viewership with a critical window into Western art history’s participation in broader socio-economic and semiotic systems. Examining artefacts from fifteenth-century painting to modern publicity, Berger’s series for the BBC explored the relational and historically contingent nature of seeing, famously stating that “perspective makes the eye the centre of the visible world.”[1]

Exposing how an era’s visual culture articulates and responds to material socio‑political dynamics, his analysis implied a necessary inverse in the artefacts and traits that go relatively unseen due to our historically contingent optics. Berger’s approach thus begs the question: How might emergent visual regimes enabled by today’s virtual reality (VR) technologies reflect a perspective that is specific to our era? And as a consequence, how might VR equally inaugurate a way of (not) seeing? This conversation is broached by contemporary artists who are testing the capacities of VR in a myriad of ways.

On a practical level, VR can be defined as a set of technologies that provide an interface for real‑time sensorimotor and cognitive activities within a digitally created artificial world.[2] These technologies establish immersive and transportive sensory environments that allow users to feel spatially present in an alternate reality. VR’s capacity to establish such a compelling virtual environment is often described in terms of degrees of immersion.[3] To achieve this, immersive systems imply a double action of both enveloping the user in a vivid and extensive alternative world, while also shutting out their physical reality.[4] This spatial chiasm can produce distinctive physiological responses in the user: from nausea, to sweating or a racing pulse, or even pain relief.

Indeed, for over a decade medical researchers have explored the benefits of VR as a non-pharmacological analgesia; the unusually high amount of attention drawn into these virtual environments serving as a neurological distraction from processing physical pain.[5] The power of escapism has been associated with historical cultural practices, including the modern fantasy genre across various media. While advocates of this style reject the reduction of this complex artistic genre to a single psychological mechanism,[6] creative researchers in VR face similar questions as to whether the technology’s immersive quality feeds a modern proclivity towards escapism, as discussed by researchers in pathological gaming.[7]

Jess Johnson and Simon Ward (Developer: Kenny Smith; Sound: Andrew Clarke), Known and Unknown, 2018, still from virtual reality installation. Exhibited as part of Terminus, 2018, at the National Gallery of Australia, Canberra. Courtesy the artist
Jess Johnson and Simon Ward (Developer: Kenny Smith; Sound: Andrew Clarke), Known and Unknown, 2018, still from virtual reality installation. Exhibited as part of Terminus, 2018, at the National Gallery of Australia, Canberra. Courtesy the artist

Artists Jess Johnson and Simon Ward are candid about the influence of gaming and the fantasy genre on their work and personal history, explaining how they provided “windows into these much greater universes than what was happening in small‑town New Zealand.”[8] This is foreshadowed in Johnson’s drawing practice, which sees monumental architectures populated with mythic humanoid figures in repeated ritualistic and symbolic forms, establishing a “generative code” for her constructed worlds.[9] Ward’s adaptation of Johnson’s surreal imagery into sensorially saturating digital worlds is done with the aim of seducing, disorienting and troubling the audience. In doing so, Johnson and Ward use VR as a psychedelic conduit that pierces the fabric of reality, indulging what they consider to be the audience’s innate exploratory drive to see beyond immediate reality.

For Johnson and Ward’s recent VR work, shown as part of the Balnaves Contemporary Intervention Series at the National Gallery of Australia, Terminus (2018), the artists developed five Head-Mounted Display (HMD) VR experiences, installing them as “stations” upon a Dungeons and Dragons inspired floor maze. Integrating Johnson’s signature iconography into the physical display, these structures read as gamified sci-fi altars, enclaves and passages, positioning the HMD VR experiences as achievements to individually unlock in a “choose-your-own‑adventure,” while a separate pavilion room allows visitors to enjoy projected animations collectively.[10] 

The impact of user immersion and agency in Johnson and Ward’s work builds upon ways of seeing that have evolved through the history of modernity. In his analysis, Jonathan Crary examines the modernisation of perceptual experience through nineteenth-century visual culture, as divertissements such as the Stereoscope engendered new forms of spectatorship. As phantasmagoria, these modern technologies foreshadowed the operations of VR, their mystifying appeal similarly functioning through “the detachment of the image from a wider field of possible sensory stimulation.”[11] 

Kaiserpanorama in Prater, Vienna, 1913. © SLUB/Deutsche Fotothek/ O. Meister
Kaiserpanorama in Prater, Vienna, 1913. © SLUB/Deutsche Fotothek/ O. Meister

The Stereoscope and later the Kaiserpanorama prefigured the contemporary HMD used in Johnson and Ward’s work, as their ways of seeing were characterised by a type of psychic and perceptual insularity.[12] Contrasting this, the immersion of the nineteenth-century panorama painting derived from its frameless, unbounded image that enabled an “impression of completeness” for the visitor to peruse along its horizontal axis.[13] Almost two centuries later, the VR HMD integrates and turbo‑charges aspects from each of these divertissements, using insular sensory stimulus that opens to an unbounded virtual image. Should we therefore conclude that VR continues the modern habituation of audiences to modes of consumption and docility, as discussed by Crary?

The agency of the viewer in Terminus has been described as “neither completely powerless nor all‑powerful” as they are “enveloped in a quest that is encompassing and transformative.”[14] Unlike the stereoscopic and panoramic immersion of the nineteenth century, the Terminus user is drawn into an alternate world that does not simulate realistic landscapes or figures, but enables a vivid and embodied experience of a speculative virtual realm. Contrary to the consumable mystique of its nineteenth-century counterparts, Johnson and Ward’s affective world represents a 21st-century site of potential agency: the infinitely plastic digital sphere where structures are built, communities formed and history made. In this way, Terminus reconfigures qualities of the postmodern fantasy genre through which real and virtual planes are intertwined as indistinguishable sides of a single mobius strip.[15]

Despite this shift, certain aspects of modern stereoscopic ways of seeing pervade VR works like Terminus. For, while the insular HMD viewing experience opens onto new virtual worlds, the embodied viewer is also an object on display. Unaware of their reactive postures and gestures, the viewer becomes a comical monument to the dissociative aspects of the VR experience. Functioning by shutting out the viewer’s physical environment, VR mystifies one aspect of the viewers reality, while engrossing them in another.

In contrast to these alternate world-building approaches, instrumental applications of VR aim to directly link experiences of the virtual space with real-world issues. This is seen in the field of VR Documentary exemplified by Clouds over Sidra (2015) produced by Gabo Arora and Chris Milk in partnership with the UN and Samsung to present a 360-degree video that immerses viewers inside the Za’atari refugee camp in northern Jordan. Developed from a journalistic tradition, such documentaries often include matter-of-fact voice‑over narration to provide context and information regarding the virtual environment in which the viewer is immersed. Jeremy Bailenson emphasises the absence of traditional emotionally intensifying filmmaking techniques in Clouds over Sidra; rather, suggesting that the film’s power arises through the viewer’s first-person visual immersion in ordinary moments within the refugee community.[16] 

The impact of VR documentary as described by Milk derives from a particular logic of visibility and proximity. The viewer not only sees ordinarily invisible corners of the world, but inhabits a perspective that suggests their embodied presence within an ordinarily distant geographic, cultural and political landscape. The efficacy of VR’s deployment in this instance is seemingly supported by the doubling of donations to the UN after the release of Clouds over Sidra.[17] Yet celebration should be paired with detailed consideration of the logic implied by this approach to engendering empathic ways of seeing.

Gab Arora and Chris Milk, Clouds Over Sidra, 2015, 360-degree video still. Produced in partnership with the United Nationa and Samsung. Photo courtesy the artist
Gab Arora and Chris Milk, Clouds Over Sidra, 2015, 360-degree video still. Produced in partnership with the United Nationa and Samsung. Photo courtesy the artist

This topic is addressed by Jeremy Bailenson, who expands studies from the cognitive sciences and psychology to consider the potential and effects of VR. Looking to Jamil Zaki’s exploration of the neural bases of social behaviour, Bailenson highlights a necessary cognitive step within empathy: “the ability of your brain to form theories about what other people are feeling and what might be causing those feelings.”[18] Arguing that empathy is switched on or off by individuals due to its emotionally taxing effects on our mental resources,[19] VR intervenes by “reliev[ing] users of the cognitive effort required to make a mental model of another person’s perspective from scratch,” providing users with a tool to “overcome a motivational hurdle.”[20] 

Echoing the title of his book, Experience on Demand, Bailenson suggests that VR can provide easier and higher‑definition[21] conditions for “perspective-taking” that encourage empathy, building upon psychological studies in this field.[22] This socially instrumental approach to VR is evident in the work of multidisciplinary collective BeAnotherLab (BAL), The Machine to be Another (2016 ‑ ongoing). The collective’s artistic aims are to use neuroscientific approaches to embodiment to explore perceptions of the Self in relation to the Other, aspiring to measure the empathy generated among its users.[23] 

Developed using low-budget Creative Commons technology, the system integrates telepresence and performance to generate a user experience of inhabiting the body of the Other ‑ one’s binary opposite in terms of gender, race and social position, among other categories. This illusion is enabled through the coupling of movements between a user and real-life performer, using head-mounted displays, headphones, microphone, head‑tracking, and servo-controlled cameras.[24] Liam Jarvis analyses his particular experience with The Machine to Be Another hosted by Good Chance Encampment, which facilitated an experience of inhabiting the virtual body of a refugee from the dismantled “jungle” refugee camp in Calais.

Looking down at his own arms that are seemingly transformed into those of a refugee, Jarvis listens to the latter’s pre-recorded account of her real-life events, while drawing a picture inspired by this story. The work concludes with the removal of his HMD, where he finds himself face-to-face with the refugee volunteer whose body he has virtually inhabited. In response to this, Jarvis deploys a Levinasian ethical framework to raise important questions about the supposed empathy implicated in this exchange, stating: “Is this illusory transaction in body ownership across not only different social, political and gender boundaries, but the borderlands of the skin, symptomatic of a radical empathic act ...? Or, should I feel a sense of unease at my perceptual colonisation of the volunteer refugee’s mediatized image?”[25] 

While acknowledging the way in which such VR experiments can provide refugees with a powerful tool to communicate their experience to others ‑ The Machine to Be Another being used not only in artistic context but also in community workshops and neurological rehabilitation contexts[26] Jarvis questions the simultaneous effacement and “possession” of the Other through the perspectival illusion of this project, through which the phenomenal self virtually integrates the Other’s physicality into their own bodily schema before meeting them in the flesh.[27] What does this gesture reveal about contemporary conditions of empathy? In a highly mediatised and informationally saturated era, does empathy require user immersion in the Other to alleviate the cognitive fatigue of imagining their perspective? Could this short cut subsequently reduce our capacity to really see the Other in the fullness of their difference?

The emotional responses elicited from The Machine to Be Another are discussed by Jarvis in relation to a “Proprioceptive drift” through which the suffering of the Other is experienced via a mislocalised sense of self, almost like a phantom limb.[28] Yet it is important to consider how the discourse surrounding projects such as Clouds over Sidra and The Machine to Be Another can become entangled with therapeutic frameworks for VR, such as the clinical treatment of anxiety disorders. For sufferers of Post-Traumatic Stress Disorder (PTSD), panic disorder, social anxiety and specific phobias, VR technology enables a controlled computer-generated environment for incremental exposure therapy.[29] 

This method provides patients with exposure in virtuo to enable their eventual tolerance of anxiety-inducing material in vivo. Advocates for VR’s social applications, while appealing to their own specific fields of neuroscientific or psychological research, rely on a similar logic of therapeutic consumption: that controlled embodied experiences of difference in virtuo will equip us for more challenging in vivo social tolerance. This begs an important question: Does the controlled therapeutic pathway to empathy in virtuo, reinforce social and political dynamics in which acceptance of the marginalised Other is framed by the needs and perspectives of the dominant?

Tabita Rezaire, PREMIUM CONNECT (REAL DEAL), 2017, virtual reality artwork commissioned by UNREAL by NRW Forum, Dusseldorf
Tabita Rezaire, Premium Connect (Real Deal), 2017, virtual reality artwork commissioned by UNREAL by NRW Forum, Dusseldorf

It is in this context that critical explorations of VR’s relationship to social, political and physical realities becomes even more important. The recent exhibition Virtual Insanity (2018) at the Kunsthalle Mainz presented the work of artists such as Cao Fei, Jon Rafman, Harun Farocki and Tabita Rezaire to explore “the extension of reality and its shadowy underbelly,”[30] by questioning how heightened immersion within today’s virtual worlds produces a profound impact on the physical world. Tabita Rezaire’s VR work, Premium Connect (Real Deal) (2016) continues her interrogation of the digital sphere as a terrain for electronic colonialism, enveloping the audience in a VR universe that intertwines cybernetic and sacred geometries, as well as computational and African divination systems. In doing so, she reminds the user of the infrastructural politics of ICT systems, their lack of neutrality and implication in the real-world erasure of Indigenous forms of knowledge.[31] 

Interrogating the spectatorship of VR technologies, artist Theo Triantafyllidis subverts VR’s structures of visibility in Staphyloculus (or the paradox of site specificity of virtual realities) (2017), producing an individual HMD interactive experience depicting the outbreak of a VR transmitted “Polywobbly Fervenitis” virus, presenting this user interaction as an supplementary spectacle for others in the space. He explains that “the whole piece is secretly choreographing the body of the person in the VR set to do weird stuff for the other people to watch, without that person necessarily noticing.”[32]Baiting its user, Triantafyllidis’ work reminds us that the febrile deployment of new visual technologies not only implies experiments with ways of seeing, but also the generation of new surveilled subjects – be they gallery audiences or potential markets.

Indeed, from a corporate perspective, the real subject of VR is the user, rather than the image enclosed in their HMD. To return to our initial question: How does VR imply both a way of seeing and way of not seeing? For artists today the technical functionalities of both saturating and shutting out visual stimulus can be harnessed in the vivification of speculative VR worlds that viscerally disturb our presence in the real world. Equally, it can provide a controlled virtual space to expose users to images that are otherwise too distant or difficult to be seen in real life. But as experimentations with this medium continue, considering what is excluded from our immersive field of vision may tell us more about our contemporary condition than what is on full display.

Theo Triantafyllidis (Sound design: Holly Waxwing; Additional photogrammetry: Regis Boissenin), Staphyloculus (or the Paradox of Site Specificity of Virtual Realities, 2017, virtual reality artwork commissioned for UNREAL by NRW Forum, Dusseldorf
Theo Triantafyllidis (Sound design: Holly Waxwing; Additional photogrammetry: Regis Boissenin), Staphyloculus or the Paradox of Site Specificity of Virtual Realities, 2017, virtual reality artwork commissioned for UNREAL by NRW Forum, Dusseldorf

Footnotes

  1. ^ Ibid. p. 1,092.
  2. ^ Philippe Fuchs & Pascal Guitton et. al. (eds), introduction to Virtual Reality: Concepts and Technologies, London: Chapman & Hall, 2011, p. 6.
  3. ^ James J. Cummings & Jeremy N. Bailenson, “How immersive is enough? Meta‑analysis of the effect of immersive technology on user presence,” Media Psychology, vol. 19, 2016, p. 3.
  4. ^ Mel Slater & Sylvia Wilbur, “A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments,” Teleoperators and Virtual Environments, vol. 6, 1997, p. 605.
  5. ^ E. Steele, K. Grimmer et. al, “Virtual Reality as a pediatric pain modulation technique,” Cyberpsychology & Behaviour, 6: 6, pp. 633–44
  6. ^ Examples include Caroline Webb, Fantasy and the Real World in British Children’s Literature, New York and London: Routledge, 2015; and Rosemary Jackson, Fantasy: The Literature of Subversion, London: Routledge, 1981.
  7. ^ Dongdong Li, Albert Liau & Angeline Khoo, “Examining the influence of actual‑ideal self‑discrepancies, depression, and escapism, on pathological gaming among massively multiplayer online adolescent gamers”, Cyberpsychology, Behavior & Social Networking, 14:9, 2011.
  8. ^ Jess Johnson quoted in A‑M Jean & E. Carlin, “Review: Constructing Fantasy Worlds at the NGA,” Art Monthly Australasia, May 2018, p. 35.
  9. ^ Interview with Jess Johnson & Simon Ward about Terminus, 2018, Balnaves Contemporary Intervention Series, National Gallery of Australia, 5 May – 24 September 2018: https://nga.gov.au/balnaves/johnsonposter.pdf.
  10. ^ “Balnaves Contemporary Intervention Series: Jess Johnson & Simon Ward, Terminus”, National Gallery of Australia: https://nga.gov.au/balnaves/johnson-ward.cfm.
  11. ^ Crary uses this word in a gesture towards Theodor Adorno’s discussion of phantasmagoria as “processes and forms that conceal their actual production and operation.” See Jonathan Crary, “Géricault, the Panorama, and Sites of Reality in the Early Nineteenth Century,” Grey Room, 9, Autumn 2002, p. 19.
  12. ^ Ibid. pp. 8–9.
  13. ^ Ibid. p. 20.
  14. ^ A‑M Jean & E. Carlin, Art Monthly Australasia, May 2018, p. 35.
  15. ^ Jeremy Bailenson, Experience on Demand, New York: W.W. Norton & Co. [Kindle DX version], p. 1,060
  16. ^ Maria Nikolajeva, “Fairy tale and fantasy: From archaic to postmodern,” Marvels & Tales, 17:1, 2003, p. 145
  17. ^ Ibid. p. 1,072.
  18. ^ Ibid. p. 1,092.
  19. ^ Ibid. p. 1,101.
  20. ^ Ibid. p. 1,143.
  21. ^ Ibid., p.1,145. Bailenson states “Because the mental model of the perspective of the empathic subject can be created in great detail in VR, it can be designed to help avoid stereotypes and false or comforting narratives.”
  22. ^ Ibid. pp. 1,123 and 1,136
  23. ^ BeAnotherLab, Machine to Be Another: themachinetobeanother.org.
  24. ^ Phillipe Bertrand, Daniel Gonzalez‑Franco et. al., “Machine to Be Another”, Proceedings of AISB 2014—50th Annual Convention of the AISB, 2014, p. 1.
  25. ^ Liam Jarvis, “The ethics of mislocalized selfhood,” Performance Research, 22: 3, 2017, p. 31.
  26. ^ Ibid., p. 32.
  27. ^ Ibid., p. 34.
  28. ^ Ibid., p. 35.
  29. ^ Brenda Kay Wiederhold & Mark D. Wiederhold, Introduction to Advances in Virtual Reality and Anxiety Disorders, 2014, p. 4.
  30. ^ See http://kunsthalle-mainz.de/en/exhibitions/archive/14.
  31. ^ IMPAKT, “Resident Artist: Tabita Rezaire”, October 2016: http://impakt.nl/headquarters/resident-artist-tabita-rezaire.
  32. ^ Theo Triantafyllidis & Faith Holland, “Queering Ork Aesthetics & Existing Beyond the Virtual”, Aqnb, July 2018: https://www.aqnb.com/2018/07/23/queering-ork-aesthetics-and-existing-beyond-the-virtual-theo-triantafyllidis-in-conversation-with-faith-holland.

Denise Thwaites is a postdoctoral fellow at the iCinema Research Centre, UNSW Art & Design, University of New South Wales.

Support independent writing on the visual arts. Subscribe or donate here.