A Self-Supervised Method for Speaker Recognition in Real Sound Fields with Low SNR and Strong Reverberation

Speaker recognition is essential in smart voice applications for personal identification. Current state-of-the-art techniques primarily focus on ideal acoustic conditions. However, the traditional spectrogram struggles to differentiate between noise, reverberation, and speech. To overcome this challenge, MFCC can be replaced with the output from a self-supervised learning model. This study introduces a TDNN enhanced with a pre-trained model for robust performance in noisy and reverberant environments, referred to as PNR-TDNN. The PNR-TDNN employs HuBERT as its backbone, while the TDNN is an improved ECAPA-TDNN. The pre-trained model employs the Canopy/Mini Batch k-means++ strategy. In the TDNN architecture, several enhancements are implemented, including a cross-channel fusion mechanism based on Res2Net. Additionally, a non-average attention mechanism is applied to the pooling operation, focusing on the weight information of each channel within the Squeeze-and-Excitation Net. Furthermore, the contribution of individual channels to the pooling of time-domain frames is enhanced by substituting attentive statistics with multi-head attention statistics. Validated by zhvoice in noisy conditions, the minimized PNR-TDNN demonstrates a 5.19% improvement in EER compared to CAM++. In more challenging environments with noise and reverberation, the minimized PNR-TDNN further improves EER by 3.71% and 9.6%, respectively, and MinDCF by 3.14% and 3.77%, respectively. The proposed method has also been validated on the VoxCeleb1 and cn-celeb_v2 datasets, representing a significant breakthrough in the field of speaker recognition under challenging conditions. This advancement is particularly crucial for enhancing safety and protecting personal identification in voice-enabled microphone applications.