UEmo-D1: An Efficient and Calibrated Decision Model for Unified Multimodal Emotion Recognition

Multimodal Emotion Recognition (MER) seeks to identify emotions conveyed through or elicited by media comprising diverse modality compositions. Research in this field has largely evolved along two paradigms: task-specific models and approaches based on Multimodal Large Language Models (MLLMs). Despite their distinct strengths, neither fully reconciles three capabilities essential to human-like emotion understanding: cross-task generalization, efficient decision-making, and receptiveness to multiple plausible interpretations. Inspired by recent advances in decision models, we explore a unified framework that maps multimodal inputs, guided by natural language instructions, directly to distributions over candidate options, addressing these requirements through direct probabilistic decisions. We instantiate this framework in UEmo-D1, the first decision model for MER to our knowledge. To support joint training and systematic evaluation, we also curate UEmo-Data, comprising 977,270 training samples and 41 evaluation subsets across six MER tasks. Extensive experiments demonstrate UEmo-D1's combination of strong recognition performance, low inference latency, and well-calibrated confidence estimates. Complementary analyses further reveal how training strategies and data composition influence these capabilities, providing practical insights into unified MER modeling. Together, our findings establish decision models as a feasible direction toward human-level emotion recognition. Project page: https://github.com/wdqqdw/UEmo-D1.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-18
DOI
https://doi.org/10.48550/arxiv.2611.24481
Primary Topic
Emotion and Mood Recognition
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

UEmo-D1: An Efficient and Calibrated Decision Model for Unified Multimodal Emotion Recognition

Runhao Zeng, Daiqing Wu, Chengyang Fang, Sicheng Zhao et al.
arXiv (Cornell University)
Emotion and Mood Recognition
preprint

UEmo-D1: An Efficient and Calibrated Decision Model for Unified Multimodal Emotion Recognition

Runhao Zeng, Daiqing Wu, Chengyang Fang, Sicheng Zhao, Fei Ma, Junqiao Wang
preprint en

Abstract

Multimodal Emotion Recognition (MER) seeks to identify emotions conveyed through or elicited by media comprising diverse modality compositions. Research in this field has largely evolved along two paradigms: task-specific models and approaches based on Multimodal Large Language Models (MLLMs). Despite their distinct strengths, neither fully reconciles three capabilities essential to human-like emotion understanding: cross-task generalization, efficient decision-making, and receptiveness to multiple plausible interpretations. Inspired by recent advances in decision models, we explore a unified framework that maps multimodal inputs, guided by natural language instructions, directly to distributions over candidate options, addressing these requirements through direct probabilistic decisions. We instantiate this framework in UEmo-D1, the first decision model for MER to our knowledge. To support joint training and systematic evaluation, we also curate UEmo-Data, comprising 977,270 training samples and 41 evaluation subsets across six MER tasks. Extensive experiments demonstrate UEmo-D1's combination of strong recognition performance, low inference latency, and well-calibrated confidence estimates. Complementary analyses further reveal how training strategies and data composition influence these capabilities, providing practical insights into unified MER modeling. Together, our findings establish decision models as a feasible direction toward human-level emotion recognition. Project page: https://github.com/wdqqdw/UEmo-D1.

arXiv (Cornell University)
Emotion and Mood Recognition
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon