On the influence of head-above-torso orientation on deep-learning-based binaural sound source localization

Abstract Although the roles of the torso and head-above-torso orientation (HATO) in human spatial hearing have been thoroughly studied in psychoacoustic research, binaural sound source localization (SSL) models generally treat head and torso as a rigid unit, not reflecting natural, independent head-torso postures or their dynamics during rotation. This study is the first to systematically assess HATO in deep-learning-based binaural SSL for both static scenes and under dynamic rotations. Convolutional recurrent networks are trained with and without varying HATO as well as with and without explicit head-orientation input (scalar angle or quaternion) across three configurations: joint head-and-torso rotation with fixed HATO, torso rotation below a fixed head, and head rotation above a fixed torso. Performance is measured via a set of complementary localization metrics including the mean great circle distance, coordinate-isolated angular components and the quadrant error rate on simulated and measured data, with analyses with respect to signal-to-noise ratio, sound-incidence direction, and head-rotation velocity. In static scenarios, matched single-HATO training reduces mean great circle distance and quadrant error rate by about 24% and 38% on the simulated dataset relative to fixed-HATO training, primarily through reductions in polar-related components and by resolving front-back confusions; training a single model across HATOs without orientation input lowers front-back confusions by more than 50% and transfers to measured data, with small additional gains from explicit input. In dynamic scenarios, torso rotation below a fixed head already yields benefits, indicating that HATO-dependent torso cues can be exploited even without varying head- and pinnae-related cues. Head rotation yields the largest improvements: errors drop rapidly by rotations with velocities up to about 24 to 30 $$^{\circ }/\mathrm{s}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mmultiscripts> <mml:mrow/> <mml:mrow/> <mml:mo>∘</mml:mo> </mml:mmultiscripts> <mml:mo>/</mml:mo> <mml:mi>s</mml:mi> </mml:mrow> </mml:math> and then plateau. Furthermore, a clear train-test mismatch is observed (variable-HATO vs. fixed-HATO), indicating that HATO-dependent torso cues are learned and that their mismatch reduces accuracy. Explicit orientation input adds gains mainly at small to moderate velocities. The largest dynamic gains occur for sources near the median plane and at elevations away from the horizontal plane, and results generalize from simulation to measurement. Overall, it is shown that modeling and exploiting HATO substantially improves deep-learning-based binaural SSL in both static and dynamic conditions.