Peer Learning via Shared Speech Representation Prediction for Target Speech Separation

Target speech separation approaches widely use the encoder-separation-decoder framework. Generally, the crucial idea of the framework is how to map the mixture of speech signals to a distinctive representation. However, existing methods fail to learn discriminative speech representations because they ignore the distinctive characteristics (tone and timbre) related to each target speaker, which are shared within the corresponding mixture samples. To acquire shared speaker characteristics and discriminative representations, we propose a novel "peer learning" method for this framework. Specifically, we construct a pair of mixture samples involving the same target speaker and design a two-branch speech representation prediction module. Then, the more discriminative representation can be determined via computing the signal-to-noise ratio and exchanged in the prediction module to improve separation performance. Experiments on the Libri2Mix dataset demonstrate the effectiveness of our method.