SoundHapticsVR: Head-Based Spatial Haptic Feedback for Accessible Sounds in Virtual Reality for Deaf and Hard of Hearing Users

Virtual Reality (VR) systems use immersive spatial audio that provide more localized sound awareness to convey critical information. However, these audio cues are often inaccessible to individuals who are Deaf or Hard-of-Hearing (DHH). To address this, we developed, a head-based haptic system designed to enable localizing and convert- ing audio signals directly into haptic feedback using multi-channel acoustic haptic actuators. We evaluated through three studies. The preliminary study determined the maximum tactile frequency threshold perceivable on different regions of the head of DHH users when a VR headset is attached. This was followed by a study to deter- mine the ideal number and arrangement of transducers for effective sound localization on the head for VR applications. Our third study focused on differentiating sound sources, where participants associated spatial sound sources with corresponding spa- tial and acoustic haptic patterns. Our findings indicate that tactile perception thresh- olds vary across different head regions, emphasizing consistent frequency equalization across all position to ensure uniform sensation. Adding a transducer to the front of the head significantly improved sound localization, serving as a critical confirmation cue for DHH users. Participants were able to correlate distinct haptic patterns with specific objects, suggesting that unique and more intense haptic patterns were helpful in localizing the contextual sound source. Overall, the system has the potential to make VR applications that has wide range of sound information to be more accessible for DHH users.