Digital Culture & AudioVisual Challenges

Summary:
The incorporation of sound design, embodied elements, and enhanced interactivity is crucial for achieving a cohesive, immersive Virtual Reality experience that is inclusive and diverse, broadening the audience and artistically diversifying the content. This article provides an overview of three case studies of Virtual Reality experiences as examples of good practice, focusing on the accessibility, interactivity, and embodied sensations that are induced through the implementation of sound. Alternative approaches, like bottom-up practices and open source design, exalt the creative opportunities to the contemporary digital tedium, renewing the promising pleasures of Virtuality.

Virtual Reality (VR) has failed as the promise of a brave new frontier, a professed land that would liberate and provide us with a breathtaking playfull, communicating and immersing field. New worlds and endless possibilities eroded to badly constructed replicas, distorted visions of a fragmented unpleasant reality. To an extent this multiplanar medium has been dictated by dispiriting rules and the access has been limited to an envisioned collective. The dream of experiencing exciting new worlds has been kept for those who could follow the digital mainstream as it has been formulated. (Ciccone et al., 2021)
Despite the increasing popularity and attention that VR has garnered in recent years, being touted as a thrilling medium for entertainment, education, training, and visualization, it appears to be confined to repeating the same limitations, with only a few instances of overcoming challenges and addressing accessibility barriers. Virtuality as it has been implemented in VR environments is under crisis; powerful industries are discussing engagement and transmedia agencies, including gaming, filming, education or healthcare, while still important misplaced links spoil the pleasure of being immersed. These include the high cost of VR equipment -including hardware and software-, the dominance of large tech companies in the market, the limited availability of quality content due to high costs of digital resources such as computational capacities and data storage, and ongoing technical issues such as tracking, resolution, and lags. Furthermore, motion sickness, this uncomfortable or even unbearable sensation of dizziness, and the lack of inclusiveness have limited further the VR audience. (Ciccone et al., 2021) It could be stated that VR environments have definitely been acclaimed as a high-valued novelty but have resulted almost as a niche technology.

On the other hand, an alternative approach is a bottom-up, open and more democratized coding -perhaps through the DIY philosophy applied to the new AI tools- that leads to dynamically changing and adapting Virtuality, enhancing the experience for an even more inclusive audience. The domain of accessibility should be a cornerstone of a successful architecture of VR environments. (Teófilo et al. 2016) Terms like accessibility and inclusiveness are not novel in technological practices; in contrast, they remain primal aims. Mott et al. encourages designers to integrate accessibility as an operational integral tool in the designing process, providing inclusiveness and better quality standards. (Mott et al., 2019) This implementation of the design methodologies occurs in tandem with the multilayer exploration of accessibility at the early stages of creation as a part of the core mechanics. By adopting this kind of strategy, the necessary space for imaginative/alternative uses, not only feel seamless but can also grant additional context variation and immersion depth. In this context, accessibility can primarily be achieved through content creation, interaction modes, and representational techniques. However, it can also be expanded to include device alternatives, such as DIY artifacts, and to the diversity of applications, including unorthodox or heretic interpretations of VR experiences. (Mott et al., 2019)

While exploring these various modalities, sound design (Rajguru et al., 2020) can be one of the pillars of the constructed experience alongside the visuals, in the same fashion that for three-dimensional vision, now there is three-dimensional sound. The sound could be molded creatively; it is more than a background soundscape that adds to the ambiance; it constitutes a fundamental building block of the virtual world´s creation. (Jain et al. 2021a) This design philosophy could be a prime opportunity to be implemented in VR experiences in order to concurrently broaden the audience and artistically diversify the content. (Friberg & Gärdenfors, 2004).
Respectively, we were drawn to works that introduce immersion by utilizing auditory stimuli in different ways, as examples of good practice. With narration and a journey through stories, Laurie Anderson & Hsin-Chien Huang's work, Chalkroom (2017), transports the audience to a realm that is reminiscent of a dream. Instead of offering an interpretation of how the visitor perceives it, the work reflects on how the visitor wanders through a virtual world of stories as the sound completes the storytelling. The Jellyfish (2021), a creation by Mélodie Mousset (HanaHana), Edo Fouillou and Christian Heinrichs, encourages the viewer to enter a surreal aquatic environment. In this VR piece, we are exposed to colorful lifeforms that are calling the participants to sing together. The luminous jellyfishes serve as the protagonists in this eerie aquatic world, a tool that participants may create and employ to form a communication model through sound. Large-scale VR project Evolver: A Virtual Reality of Life and Breath, by Marshmallow Laser Feast (2022), is a work that was developed in an effort to envelop us in a surreal journey into the depths of the human experience. It emphasizes the idea of breath, life and nature, interconnecting each of those elements with the cycle of respiration, a function that is given an ontological dimension through an intense soundscape.

The incorporation of sound design, narration, and interactive sound elements is crucial for achieving a cohesive, immersive virtual reality experience that is inclusive and diverse, broadening the audience and artistically diversifying the content. (Jain et al. 2021b) VR environments to some extent have already made strides in the field of visual fidelity and graphics as a means of both accessibility and diversity. VR is an all-sensorial /full-body endeavor that complements each individual function to achieve the goal of a cohesive narration in an embodied manner. It is crucial to challenge the established norms in order to rethink and redesign more inclusive and varied procedures. Heretic, erroneous uses and applications, surrealistic worlds and embodied sensations, interactive inducing soundscapes and multiplanar spatial sound present a vast territory that can restore pleasures, thrill and immersive adventures in VR environments.

References:
Ciccone Brendan, Bailey Shannon and Lewis Joanna. (2021). ”The Next Generation of Virtual Reality: Recommendations for Accessible and Ergonomic Design”. Ergonomics in Design: The Quarterly of Human Factors Applications. 31. Pp. 106480462110025. doi: 10.1177/10648046211002578.

Friberg Johnny & Gärdenfors Dan. (2004). “Audio Games: New perspectives on game audio”. Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, 2004, Singapore. pp. 148-154.

Jain Dhruv, Junuzovic Sasa, Ofek Eyal, Sinclair Mike, Porter John, Yoon Chris, Machanavajhala Swetha, and Morris Meredith. (2021, a). “A Taxonomy of Sounds in Virtual Reality”. ACM Designing Interactive Systems (DIS2021). pp. 160-170. doi:10.1145/3461778.3462106

Jain Dhruv, Junuzovic Sasa, Ofek Eyal, Sinclair Mike, Porter John, Yoon Chris, Machanavajhala Swetha and Morris Meredith. (2021, b). “Towards Sound Accessibility in Virtual Reality”. ICMI '21: International Conference on Multimodal Interaction. pp. 80-91. doi:10.1145/3462244.3479946.

Mott Martez, Cutrell Edward, Gonzalez Franco Mar, Holz Christian, Ofek Eyal, Stoakley Richard and Ringel Morris Meredith. (2019). "Accessible by Design: An Opportunity for Virtual Reality" IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) Beijing, China. 2019. pp. 451-454. doi:10.1109/ISMAR-Adjunct.2019.00122.

Rajguru Chinmay, Obrist Marianna and Memoli Gianluca. (2020). “Spatial Soundscapes and Virtual Worlds: Challenges and Opportunities”. Frontiers in Psychology. pp 1-9. Vol 11. Article 569056. doi: 10.3389/fpsyg.2020.569056.

Teófilo Mauro, Nascimento Josiane, Santos Jonathan, Albuquerque Yves, Souza, Andre and Nogueira Daniel. (2016). “Bringing basic accessibility features to virtual reality context”. IEEE Virtual Reality (VR) Conference. pp. 293-294. doi:10.1109/VR.2016.7504769.