Musical Performances in Virtual Reality with Spatial and View-Dependent Audio Descriptions for Blind and Low-Vision Users

Virtual reality (VR), inherently reliant on spatial interaction, poses significant accessibility barriers for individuals who are blind or have low vision (BLV). Traditional audio descriptions (AD) typically provide a verbal explanation of visual elements in 2D or flat video media, facilitating access for BLV audiences but failing to convey the complex spatial information essential in VR. This shortfall is especially pronounced in musical performances, where understanding the spatial arrangement of the stage setup and movements of performers is crucial. To overcome these limitations, we have developed two AD approaches—Spatial AD for a dance performance and View-dependent AD for an instrumental performance—within VR-based 360° environments. Spatial AD employs spatial audio technology to align descriptions with corresponding visuals, dynamically adjusting to follow the visuals, such as the movements of performers in the dance performance. Meanwhile, View-dependent AD adapts descriptions based on the orientation of the VR headset, activating when particular visuals enter the central view of the camera, ensuring that the description aligns with the user’s attention directed to a particular location within the VR environment. These methods are designed as enhancements to traditional AD, aiming to improve spatial orientation and immersive experiences for BLV audiences. This demonstration showcases the potential of these AD approaches to improve interaction and engagement, furthering the development of inclusive virtual environments.