TDGN: A text-guided dual-gated network for multimodal sentiment analysis

Multimodal Sentiment Analysis (MSA) aims to interpret emotions by integrating textual, acoustic, and visual information. However, the heterogeneous quality and weak correlation among nonverbal modalities often lead to unstable alignment and ineffective fusion. To address these challenges, we propose a Text-Guided Dual-Gated Network (TDGN) that introduces a hierarchical gating and text-anchored contrastive learning framework. During the alignment phase, a Text-Anchored Gated Attention (TGA) module employs text as a semantic anchor to guide fine-grained alignment between audio and visual modalities while suppressing noise and emphasizing salient emotional cues. For the fusion stage, a Dual-layer Gated Fusion (DGF) module performs intra-modal gating to refine modality-specific features and inter-modal gating to dynamically balance cross-modal contributions. Furthermore, we introduce a Text-Anchored Contrastive Learning (TACL), which constructs contrastive targets based on textual similarity anchors, ensuring the fused features maintain both modal consistency and modal diversity. Extensive experiments on the CMU-MOSI and CMU-MOSEI benchmarks demonstrate that TDGN achieves state-of-the-art performance, achieving higher accuracy and robustness. Ablation studies and visualization further validate the effectiveness of each component.

Authors

Institutions

Publication Details

Journal
PLoS ONE
Published
2026-05-12
DOI
https://doi.org/10.1371/journal.pone.0349024
Primary Topic
Emotion and Mood Recognition
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

TDGN: A text-guided dual-gated network for multimodal sentiment analysis

Yangshuyi Xu, Wenyan Xiao, Lin Zhang
PLoS ONE
Emotion and Mood Recognition
article

TDGN: A text-guided dual-gated network for multimodal sentiment analysis

Yangshuyi Xu, Wenyan Xiao, Lin Zhang
article en

Abstract

Multimodal Sentiment Analysis (MSA) aims to interpret emotions by integrating textual, acoustic, and visual information. However, the heterogeneous quality and weak correlation among nonverbal modalities often lead to unstable alignment and ineffective fusion. To address these challenges, we propose a Text-Guided Dual-Gated Network (TDGN) that introduces a hierarchical gating and text-anchored contrastive learning framework. During the alignment phase, a Text-Anchored Gated Attention (TGA) module employs text as a semantic anchor to guide fine-grained alignment between audio and visual modalities while suppressing noise and emphasizing salient emotional cues. For the fusion stage, a Dual-layer Gated Fusion (DGF) module performs intra-modal gating to refine modality-specific features and inter-modal gating to dynamically balance cross-modal contributions. Furthermore, we introduce a Text-Anchored Contrastive Learning (TACL), which constructs contrastive targets based on textual similarity anchors, ensuring the fused features maintain both modal consistency and modal diversity. Extensive experiments on the CMU-MOSI and CMU-MOSEI benchmarks demonstrate that TDGN achieves state-of-the-art performance, achieving higher accuracy and robustness. Ablation studies and visualization further validate the effectiveness of each component.

PLoS ONEVol. 21(5)
Shanghai Maritime University (CN)
National Natural Science Foundation of China
Openalex Percentile: Top 4%
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

TDGN: A text-guided dual-gated network for multimodal sentiment analysis — Yangshuyi Xu, Wenyan Xiao, et al. · PLoS ONE (2026) | TGRS Research Map | TGRS