Multimodal Text and Image Social Emotion Analysis

With the rise of social media, multimodal emotion analysis has become a research hotspot. Despite the progress made in the fusion of multimodal data by existing studies, challenges remain: current models often overlook the similarities between text and vision, fail to effectively handle noise in multimodal data, and do not sufficiently consider the contributions of different modalities, which limits the generalization and accuracy of the models. To address these issues, this study proposes an innovative multimodal emotion analysis model (EMHA-MMEA). The model combines an emotion dictionary with a cross-modal multi-head attention mechanism to enhance the model's understanding of the complex relationships between multimodal data. Compared to traditional models, EMHA-MMEA can more effectively integrate textual and visual features, focusing on emotional features between modalities, thereby more accurately capturing emotional tendencies. Experimental results on the MVSA-Single dataset show that EMHA-MMEA's accuracy in predicting emotional labels on the test set has increased by 4% compared to the baseline model, proving its effectiveness and practicality. This achievement provides technical support for the fields of social media emotion analysis and public opinion monitoring, and offers new ideas for future research.