A Cross-Modal Fusion Network with Reinforcement Learning for Emotion Recognition in Conversations

Multimodal Emotion Recognition in Conversation has garnered increasing interest in recent times. Unlike unimodal ERC, multimodal ERC leverages complementary information across different modalities. However, existing studies often fail to fully leverage rich collaborative relationships across modalities. Additionally, most previous ERC models focus solely on semantic-level context modeling, while ignoring the emotion-dependent context information. In this work, we introduce a Cross-Modal Fusion network with Reinforcement Learning (CMF-RL), consisting of three core components: a unimodal encoder (GRUEnc), a cross-modal fusion module (ComAttn), and a reinforcement learning module (RL). GRUEnc extracts contextual emotion cues at the utterance level; ComAttn integrates multimodal information by capturing intra-modal and inter-modal collaborative relationships between multiple modalities; RL module enhances emotional-level context modeling by capturing explicit emotional transference in a continuous emotion flow across utterances. Experiments on IEMOCAP and MELD demonstrate that our CMF-RL significantly improves ERC performance, outperforming state-of-the-art models.