Multimodal Feedback for Effective Takeover in Automated Vehicles for Hearing Impairment

Most vehicles on the road today range from SAE Level 1 to 3 automated vehicles, where Level 3 automated vehicles can handle most of the driving tasks on the road. Nevertheless, there are still moments when drivers need to prepare to take over the vehicle, such as when an object suddenly appears in the middle of the road. To respond to these critical events in time, multimodal displays have been introduced in instructional and informational formats. Yet, the effects of multimodal displays during takeovers for people with hearing impairments have yet to be studied. To address this, we investigated how signal type (single-modal vs. multimodal), informational type (instructional vs. informational), and hearing impairment (hearing-impaired and non-hearing-impaired drivers) can impact drivers’ takeover performance. These study findings can be used when implementing multimodal displays in automated vehicles for drivers with hearing impairments.