Multilayer Feature Fusion and Joint Loss Optimization for Emotion Recognition in Conversations

The goal of Emotion Recognition in Conversations (ERC) is to accurately identify the emotions expressed in each utterance within a dialogue. Despite advancements made by current ERC methods, particularly those using RNN-based and GCN-based models to capture emotional dynamics and model speaker relationships, there remain two primary limitations: first, an insufficient integration of multiple feature representations and commonsense knowledge, which hampers the model's ability for deep emotional understanding; and second, the reliance on a single cross-entropy loss for classification optimization, which restricts the accuracy and robustness of emotion recognition. To address these issues, we propose a method for ERC using Multilayer Feature Fusion and Joint Loss Optimization (MFFJL). This approach combines contextual information, speaker dependency, and commonsense knowledge features by extracting feature vectors through RoBERTa and COMET, utilizing bidirectional LSTM and attention mechanisms to capture conversational context, and applying a cross-fusion module to deeply integrate various features, thus enhancing comprehension of complex emotional expressions. Additionally, the feature classification module incorporates joint cross-entropy and KL divergence optimization, further improving classification accuracy and consistency. Experimental results demonstrate the effectiveness of our method, as evidenced by superior performance on the IEMOCAP and MELD datasets. Our code is available at https://anonymous.4open.science/r/MFFJL-D9D8.