Effect of HRTFs and head motion on auditory-visual localization in real and virtual studio environments

When localizing sound sources in natural or virtual environments, multisensory integration that combines auditory and visual cues is often involved. This study aims to determine the effect of (non-)individual head-related transfer functions (HRTFs) on sound source localization in the presence of visual cues. Participants (N = 16) were seated in a hemispherical loudspeaker arrangement, and their task was to indicate the perceived direction of pink-noise bursts. The conditions included localization of real loudspeakers and virtualized loudspeakers based on binaural rendering with (non-)individual HRTFs. The horizontal spacing of the visible loudspeakers was 30 degrees or larger and the vertical spacing was varied between 30 degrees and 15 degrees, to focus on vertical discrimination. The results of a static localization experiment (no head movements during playback) demonstrated an increased rate of vertical local confusions using non-individual binaural room impulse responses (BRIRs) of the KU100 dummy head compared to individual BRIRs. Dynamic auditory-visual localization with non-individual HRTFs suffered from vertical uncertainty and bias in the frontal area, where local confusion rates were consistent with the static experiment. In conclusion, the horizontal localization mapping of dynamic binaural rendering with non-individual HRTFs was comparable to real loudspeaker reproduction, while the vertical mapping was significantly impaired.