Multimodal Sentiment Analysis Based on Hierarchical Distance-Aware Contrastive Learning

多模态情感分析(multimodal sentiment analysis, MSA)利用视觉、文本和音频等模态数据来提升情感分析的准确性。尽管多模态信息能够提供更丰富的语境，但如何有效地处理异构模态数据之间的交互与融合仍然是一个重要挑战。为了解决这一问题，本文提出了一种基于分层距离感知对比学习(hierarchical distance-aware contrastive learning, HDACL)的多模态情感分析方法。具体而言，HDACL通过引入跨模态注意力机制，实现了不同模态数据之间的充分交互。与此同时，我们设计了一种基于情感强度距离差异引导的对比学习策略，进一步增强了多模态数据的一致性对齐。在CMU-MOSI多模态情感分析数据集上进行验证，实验结果表明，HDACL方法在Acc-2和Acc-7指标上分别取得了0.7%和0.8%的性能提升。Multimodal sentiment analysis (MSA) utilizes visual, textual, and audio data to improve the accuracy of sentiment analysis. Although multimodal information can provide richer context, how to effectively handle the interaction and fusion across heterogeneous multimodal data remains an important challenge. To this end, this paper proposes a multimodal sentiment analysis method based on hierarchical distance-aware contrastive learning (HDACL). Specifically, HDACL achieves full interaction across different modal data by introducing a cross-modal attention mechanism. Meanwhile we design a contrastive learning strategy guided by the difference in sentiment intensity distance to further enhance the consistency alignment of multimodal data. The method was validated on the CMU-MOSI multimodal sentiment analysis dataset. Experimental results show that the HDACL method achieved 0.7% and 0.8% performance improvements on the Acc-2 and Acc-7 indicators, respectively.