Supervised contrastive probability learning and sharpness-aware minimization model for multimodal emotion recognition in conversations

Multimodal Emotion Recognition in Conversation (MERC) aims to extract emotional information from conversations, but its performance is limited by the quality of the dataset. In real life, data often exhibits data imbalance problem. Current solutions to this problem, such as data resampling and category-balanced loss functions, have their limitations. This paper proposes a model named SCPL-SAM, which combines supervised contrastive probability learning and sharpness-aware minimization techniques to effectively address the data imbalance issue. By modeling the sample feature distribution with Angular Central Gaussian distribution for data augmentation, the model reduces the number of samples and computational resources required for contrastive learning, and encourages the model to escape “saddle points,” thereby enhancing classification performance. On the MELD and IEMOCAP datasets, the SCPL-SAM model outperforms other models, demonstrating its effectiveness in dealing with the data imbalance issue.