A unified deep learning framework for estimating acoustic context parameters from first order ambisonic speech recordings

Abstract Estimating acoustic context parameters is essential for characterizing acoustic environments, thereby enhancing immersive perception in spatial audio creation and improving speech enhancement and dereverberation algorithms. In this paper, we propose a unified deep learning based framework that estimates various acoustic contexts, including frequency-dependent reverberation time ( $$T_{30}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>T</mml:mi> <mml:mn>30</mml:mn> </mml:msub> </mml:math> ), direct-to-reverberant ratio, clarity ( $$C_{50}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:msub> <mml:mi>C</mml:mi> <mml:mn>50</mml:mn> </mml:msub> </mml:math> ), room geometry, and sound source orientation from first-order Ambisonics (FOA) speech recordings. Our framework employs a novel feature, termed the Spectro-Spatial Covariance Vector (SSCV), which efficiently represents the temporal, spectral, and spatial information of FOA signals. This feature can be effectively utilized by several deep neural networks as back-ends. Experimental results demonstrate that the proposed framework, which incorporates spatial information derived from FOA recordings, significantly outperforms existing methods based solely on spectral information from single-channel audio, achieving more than a 50% reduction in estimation error across all acoustic context estimation tasks. Additionally, we introduce FOA-Conv3D, a novel back-end network that effectively utilizes the SSCV feature through a 3D convolutional encoder. FOA-Conv3D outperforms currently widely applied deep learning frameworks such as convolutional neural network and recurrent convolutional neural network back-end architectures in acoustic parameter and orientation estimation tasks, exhibiting greater robustness under both pink and babble noise conditions. Finally, ablation studies reveal the relative contributions of spectral, interaural level difference, and interaural phase difference cues within the SSCV representation.