A learning-based model matching framework for tunable global audio telepresence

This study presents a Learning-based Tunable GlObal Audio Telepresence (LT-GOAT) system to bring the near end users to the virtual far end, with a tunable balance between the target source field and the ambient field consisting of interference and late reverberation for acoustic array systems. The system can seamlessly vary between “Enhanced” mode for speech quality and “Immersive” mode for spatial ambience. The far-end audio signals are captured by a microphone array, and then a neural network is pre-trained to separate the target source field and ambient sound. Another network is then implemented to perform multichannel filtering based on the model matching principle. The training process minimizes the matching error between the synthesized sound field and the desired sound field at the control points situated virtually at the far end. Simulations were conducted for the LT-GOAT system involving a four-microphone array at the far end and a six-loudspeaker array in the near end. A target source and an interferer at the far end were studied under various reverberation conditions. Performance was assessed using objective metrics including Perceptual Evaluation of Speech Quality (PESQ), Short-Time Objective Intelligibility (STOI) and matching errors. The results showed that the proposed system outperformed conventional baselines.