Reliability-Aware Multi-View Fusion for Robust Multimodal Sarcasm Detection with Incomplete Observations

Multimodal sarcasm detection identifies ironic intent by jointly analyzing text and images. It has attracted increasing attention due to its importance in understanding user-generated content on social media. However, multimodal observations are often incomplete due to data loss during transmission or collection, leading to unreliable predictions in real-world multimodal sarcasm detection scenarios. To address incomplete observations, we further propose a Reliability-Aware Dynamic Fusion (RADF) module, which predicts the reliability of the textual, visual, and interactive views from their representations, converts these reliability scores into dynamic fusion weights via a temperature-scaled softmax to control the sharpness of the weight distribution, and refines the fused features through feature-wise scaling. In this way, degraded views are suppressed while more informative views are emphasized under incomplete observations. Extensive experiments on public datasets validate the effectiveness of our approach, which consistently outperforms existing baselines.

Authors

Institutions

Publication Details

Journal
ACM Transactions on Multimedia Computing Communications and Applications
Published
2026-06-18
DOI
https://doi.org/10.1145/3820382
Primary Topic
Sentiment Analysis and Opinion Mining
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Reliability-Aware Multi-View Fusion for Robust Multimodal Sarcasm Detection with Incomplete Observations

Chao Kong, Sanmin Liu, 更治 筏, Junjie Chen et al.
ACM Transactions on Multimedia Computing Communications and Applications
Sentiment Analysis and Opinion Mining
article

Reliability-Aware Multi-View Fusion for Robust Multimodal Sarcasm Detection with Incomplete Observations

Chao Kong, Sanmin Liu, 更治 筏, Junjie Chen, Subin Huang, Hao Guo
article en

Abstract

Multimodal sarcasm detection identifies ironic intent by jointly analyzing text and images. It has attracted increasing attention due to its importance in understanding user-generated content on social media. However, multimodal observations are often incomplete due to data loss during transmission or collection, leading to unreliable predictions in real-world multimodal sarcasm detection scenarios. To address incomplete observations, we further propose a Reliability-Aware Dynamic Fusion (RADF) module, which predicts the reliability of the textual, visual, and interactive views from their representations, converts these reliability scores into dynamic fusion weights via a temperature-scaled softmax to control the sharpness of the weight distribution, and refines the fused features through feature-wise scaling. In this way, degraded views are suppressed while more informative views are emphasized under incomplete observations. Extensive experiments on public datasets validate the effectiveness of our approach, which consistently outperforms existing baselines.

ACM Transactions on Multimedia Computing Communications and Applications
Anhui University (CN), Anhui Polytechnic University (CN)
Openalex Percentile: Top 4%
Sentiment Analysis and Opinion Mining
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.