MESA: A Multimodal Sentiment Analysis Model Integrating Multi-Expert Mechanism and Enhanced Semantic Attention

With the increasing prevalence of user-generated content on social media platforms, multimodal sentiment analysis (MSA) has emerged as a critical research area, aiming to jointly interpret textual and visual information to understand users' emotional states. However, existing fusion strategies often struggle with modality misalignment and redundant feature representa-tions, which degrade model performance. To overcome these challenges, this paper proposes MESA, a multimodal sentiment analysis model that integrates an Enhanced Semantic Attention (ESA) module and a Multi-Expert fusion mechanism guided by uncertainty. Built upon BERT for text and ResNet50 for images, MESA enhances semantic alignment across modalities and dynamically fuses modality-specific features based on uncertainty cues. Experiments on MVSA-Single and MVSA-Multiple datasets show that MESA significantly outperforms several strong baselines in terms of Accuracy and F1-Avg, demonstrating its effectiveness and robustness.

Authors

Institutions

Publication Details

Published
2025-05-16
DOI
https://doi.org/10.1145/3757110.3757171
Primary Topic
Topic Modeling
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

MESA: A Multimodal Sentiment Analysis Model Integrating Multi-Expert Mechanism and Enhanced Semantic Attention

Lihua Huang, Xing Lyu, Jingwei Zhang, Yinping Zhang et al.
Topic Modeling
article

MESA: A Multimodal Sentiment Analysis Model Integrating Multi-Expert Mechanism and Enhanced Semantic Attention

Lihua Huang, Xing Lyu, Jingwei Zhang, Yinping Zhang, T. C. Yu
article en

Abstract

With the increasing prevalence of user-generated content on social media platforms, multimodal sentiment analysis (MSA) has emerged as a critical research area, aiming to jointly interpret textual and visual information to understand users' emotional states. However, existing fusion strategies often struggle with modality misalignment and redundant feature representa-tions, which degrade model performance. To overcome these challenges, this paper proposes MESA, a multimodal sentiment analysis model that integrates an Enhanced Semantic Attention (ESA) module and a Multi-Expert fusion mechanism guided by uncertainty. Built upon BERT for text and ResNet50 for images, MESA enhances semantic alignment across modalities and dynamically fuses modality-specific features based on uncertainty cues. Experiments on MVSA-Single and MVSA-Multiple datasets show that MESA significantly outperforms several strong baselines in terms of Accuracy and F1-Avg, demonstrating its effectiveness and robustness.

Guilin University of Electronic Technology (CN)
Openalex Percentile: Top 85%
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.