Assessing speech comprehension ability and hearing aid benefit using realistic audio-visual conversation materials

There is a lack of sensitive assessments that can be used to reliably evaluate and optimize the real-world benefit of hearing devices in noisy, multi-talker environments. We have therefore developed a real-time speech comprehension test that is based on the word-monitoring paradigm and utilizes short audio-visual recordings of spontaneous two-talker conversations on different topics (e.g., “seasons”). The participant’s task is to press a hand-held button each time they hear a target word related to the topic (e.g., “summer”). Their performance is measured by the accuracy in identifying target words (i.e., hit rate) and associated response times. The conversation recordings were presented in a realistic café and dinner party environment using a 41-channel loudspeaker array and a cylindrical video projector system. Additionally, the participant's head- and eye movements were recorded using Tobii eye-tracking glasses and a Vicon motion-tracking system. We tested 18 adults with moderate to moderate-severe hearing loss who were fitted with research hearing aids that provided different speech enhancement strategies, including an adaptive bilateral beamformer and a DNN-based algorithm. Results revealed that with increasing task difficulty (i.e., environment noise level) the hit rate decreased and response time increased. Overall, the DNN-based algorithm resulted in the highest hit rates and shortest response times.