Deep informed spatio-spectral filtering for multi-channel speech extraction against steering vector uncertainties

Adaptive beamforming combined with post-filtering is one of the most widely used techniques in suppressing directional interference and environmental ambient noise , as well as reverberation . However, many adaptive beamforming methods are often relatively sensitive to the steering vector mismatch, and their performance degrades a lot for practical applications, although pioneer researchers have made great efforts on improving the robustness. To achieve better performance in challenging scenarios, this paper proposes a two-stage deep informed spatio-spectral filtering for multi-channel speech extraction, which removes interference, noise, and reverberation simultaneously when the steering vector error exists. In the first stage, a direction-informed dual-path beamforming network was introduced to extract the target directional speech with only its early reflections. To improve the robustness, an information rectification block was designed to compensate for the signal model mismatch, and the steering vector uncertainty was taken into account in the training phase. Besides, a dual-path beamforming module was adopted to reduce magnitude distortion and improve phase recovery simultaneously. In the second stage, a magnitude-phase fusion network was proposed, serving as the post-processing module to further fuse the magnitude and phase estimated by the first stage. Experimental results confirmed that the proposed method was more robust to the signal model mismatch and achieved better performance than other baseline methods in terms of speech quality and intelligibility. • A dual-path neural beamforming and fusion system is proposed for speech extraction. • Information rectification embedding module is designed to overcome steering vector uncertainties. • The proposed system effectively rectifies the steering vector error problem. • The performance of the proposed system outperforms existing state-of-the-art baselines. • The performance of the proposed system is robust to real-world scenarios.