Enhancing in-car interaction: Analyzing Deaf and Hard-of-Hearing Drivers’ Usage, and Interaction with Voice-controlled In-car Infotainment System

Voice Control Interface is a technology that allows users to use spoken language to interact with a system. The system is capable of speech recognition where the user can input a command, the system understands and executes the commands, and gives an output through text and/or speech. Voice Control Interfaces (VCIs) have become an ubiquitous technology, from public information kiosks to personal computer devices and intelligent virtual assistant devices. The technology is constantly evolving and improving with the intentions to make life easier, however, research has revealed unique challenges VCIs pose for Deaf and hard-of-hearing (DHH) users due to reliance on speech as input and output. Despite extensive research on VCIs in personal devices and home settings, there’s a gap in understanding the specific challenges of VCIs in vehicles. The research aims to understand the challenges drivers face interacting with VCIs in vehicles and investigate the accessibility of the systems specifically in the context of its speech interface. To address this, we conducted a survey with 193 DHH and hearing participants, followed by interviews with 8 DHH and 8 hearing participants to explore their experiences and preferences regarding alternative interaction methods for in-vehicle VCIs. Results from the interview and survey reveal technical, linguistic, and environmental challenges affecting VCI efficiency in vehicles for drivers. Additional challenges were highlighted by DHH drivers due to the system’s reliance on spoken language highlighting the need for alternative approaches. A follow-up contextual inquiry with DHH participants suggests using gesture-based input and one-handed sign language as viable alternatives for in-vehicle VCIs, and highlighting feasible gesture and sign language-based interactions with the in-vehicle systems.