Deep Neural Network Based Adaptive Beamforming for Real-Time Speech Enhancement

This article introduces a deep-learning driven adaptive beamforming designed for real-time speech enhancement in hands-free headset communication, utilizing multiple microphones. To overcome challenges related to unknown speech source directions, the proposed system employs an innovative configuration of microphone arrays and robust steering vector analysis. The core assumption is that the beamformer, guided by the steering vector of the current frame, can effectively direct subsequent frames toward the speech source. The methodology integrates a speech mask-based Minimum Variance Distortion-less Response (MVDR) beamforming technique with a dual-signal transformation LSTM network (DTLN), first presented in the Deep Noise Suppression (DNS) Challenge. In this work, beam-forming is implemented directly in the frequency domain of the DTLN, streamlining the process and further enhancing real-time performance. This frequency-domain approach capitalizes on the inherent granularity for superior noise suppression and speech separation. The system employs a layered network combining short-time Fourier transform (STFT) with a learned analysis and synthesis basis to estimate speech masks in noisy environments. The estimated mask is then used to construct a beamformer focused on the speech signal while reducing background noise. By leveraging two signal transformations, the method enhances the DTLN's ability to extract both magnitude and phase information effectively. Experimental results show that the proposed system outperforms existing speech enhancement techniques in improving speech clarity and noise reduction on noisy speech datasets, offering significant advancements in hearing aid technology and communication in noisy environments.