Multi-Stage Confidence-Guided Diffusion and Emotional Bidirectional Mamba for Robust Speech Emotion Recognition

Speech Emotion Recognition (SER) in noisy environments is challenging due to the overlap between emotional and noise-related signals. We propose a novel emotion-diffusion approach to enhance SER performance in noisy conditions that transfers emotional information from clean to noisy speech through a multi-stage diffusion process. First, Mel Frequency Cepstral Coefficients and their delta features are extracted to capture emotional dynamics. Additionally, an Emotional Bidirectional Mamba Encoder with a Multi-time View Bidirectional State Space Model is designed to capture temporal emotional patterns. Next, the Emotion-Transferring Diffusion Network (ETDN) applies confidence filtering to retain key emotional features, ensuring effective emotion transfer despite noise. Finally, the Confidence-Guided Mutual Learning strategy refines noisy features for the Classification Network (CN), while the CN supervises the ETDN to maintain label consistency. Experiments on the IEMOCAP dataset show significant improvements in weighted accuracy, unweighted accuracy, and weighted F1 score across various signal-to-noise ratios. b Our code and model are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Littleznnz/ETDN.</uri>