Target-direction sound extraction using a hybrid DSP/deep learning approach

The goal of Target Sound Extraction (TSE) is to isolate sound signals of interest from a mixture of overlapping sounds. In learning-based TSE approaches, the cues, such as sound embedding, which provide information about the sound characteristics, are used as auxiliary features for the Deep Neural Networks (DNNs) to extract the target sound signals. However, in the presence of strong interference, the extraction performance is reduced when the model relies solely on the sound embedding. To this end, this study proposes an efficient Target-Direction Sound Extraction (TDSE) system by integrating DSP-based spatial feature extraction and a DNN backend. In spatial feature extraction, the spatial blocking technique is used to analyze the signals out of the target direction. The DNN backend is based on the Waveformer backbone, which has shown promising results and real-time capability on single-channel TSE tasks. Experimental results show that the proposed TDSE system can achieve 2–3 dB improvement over the Waveformer baseline. In addition, the proposed system performs well on a variety of array setups, including those not included in the training set.