Enhancing Speech Emotion Recognition With Conditional Emotion Feature Diffusion and Progressive Interleaved Learning Strategy

Speech emotion recognition (SER) aims to identify the speaker's emotional states in specific utterances accurately. However, existing methods still face feature confusion when attempting to recognize certain emotions because traditional acoustic feature extraction methods fail to capture dynamic emotional changes, blurring emotional boundaries. Additionally, existing classification networks (CNs) are constrained by fixed learning strategies, hindering their ability to capture subtle emotional nuances and resulting in label confusion. To address these two issues, we introduce 3D multiresolution modulation filtered cochleogram (MMCG) features by computing the deltas and delta-deltas of MMCG features to enhance the dynamic emotional changes and produce distinct emotional boundaries. We then customize a conditional emotion feature diffusion (CEFD) module, which progressively diffuses features based on emotional context to retain emotional nuances effectively and reduce reliance on conditioned information. In addition, a confidence filtering module is used to filter diffused features based on confidence-based posterior probabilities to ensure enhanced feature discrimination. We design a flexible training strategy named the progressive interleaved learning strategy (PILS) to learn further complex emotional nuances, which consists of two alternating stages: fine-tuning the CN parameters and supervising the CEFD output. Testing on the IEMOCAP, CASIA, and EMODB corpora demonstrates significant performance improvements in SER.