Sensing the unseen: innovative haptic and audio approaches for inclusivity in virtual reality

Cinematic virtual reality has the transformative potential to create immersive storytelling experiences through the integration of dynamic audio-visual elements. However, for blind and visually impaired audiences, this potential is often constrained by inadequate accessibility features, particularly in audio description, which traditionally narrates visual content. The fast-paced and interactive nature of virtual reality frequently outpaces conventional audio-description methods, resulting in a diminished experience for these users. ‘Virtual Reality for the Visually Impaired’ addresses this challenge by developing the concept of ‘description spheres’ – virtual objects embedded with layers of information that combine immersive audio, haptic feedback, sound effects, and advanced audio description techniques. These virtual spheres, which activate upon touch, can be attached to any virtual-reality object, providing users with more detailed contextualisation and crucial spatial information such as position, direction, and distance. This innovation enriches the narrative experience by offering an enhanced and tangible perception of the virtual-reality environment. Additionally, the concept of ‘audio introductions’ is employed to describe key visual elements before the virtual-reality experience begins. Supported by three-dimensional audio and experimental audio description, these introductions help reduce cognitive load and improve immersion, allowing users to form a clearer mental image of their surroundings and the narrative. This method promotes deeper engagement with the virtual reality content by facilitating a more comprehensive understanding of the environment and story. By integrating these multi-sensory enhancements and collaborating with the Royal National Institute for Blind People in the United Kingdom, our research aligns with contemporary advancements in immersive sound and visualisation, and this chapter explores how such hybrid-modality description spheres can enhance virtual-reality accessibility. It also contributes insights into the broader discourse on translating sensory experiences into accessible formats, advancing the discussion on inclusive practices in virtual reality and beyond.