Context-Adaptive Affective Behavior and Artistic Portrait Animation for AIGC Virtual Humans via Temporally Structured Action Units

Affective virtual humans require facial responses that evolve coherently over time, reflect conversational context, and transfer across heterogeneous character appearances. Existing emotional portrait-animation methods commonly rely on discrete emotion labels, frame-wise motion variables, or latent affect representations. This study formulates AIGC-supported virtual-human animation as a context-conditioned temporal affective-behavior modeling problem and proposes TABI-DH. The framework integrates multimodal speech and context encoding, a Temporal AU Behavior Planner, AU-guided non-rigid facial geometry, externally controlled rigid head pose, and behavior–appearance factorized rendering. On MEAD, TABI-DH achieves 22.74 dB PSNR, 0.71 SSIM, 84.25% emotion accuracy, and an AU-trajectory correlation of 0.89. Cross-dataset evaluation on LRW, GRID, and HDTF demonstrates consistent performance under distribution shift. In session-independent IEMOCAP evaluation, incorporating past conversational information increases four-class emotion agreement to 77.1% and reduces affective-response onset error to 176 ms. Mechanism analyses indicate that temporal AU constraints and upper-face behavior primarily improve affective fidelity rather than lip synchronization. Artistic-portrait experiments further show that a common AU trajectory can transfer across photographic and stylized identities while preserving source appearance. These findings support temporally structured AUs as an interpretable representation connecting affective interaction modeling with controllable AIGC virtual-human animation.

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-18
DOI
https://doi.org/10.21203/rs.3.rs-11182568/v1
Primary Topic
Human Motion and Animation
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Context-Adaptive Affective Behavior and Artistic Portrait Animation for AIGC Virtual Humans via Temporally Structured Action Units

Yanshuai Shi
1 links
Research Square
Human Motion and Animation
preprint

Context-Adaptive Affective Behavior and Artistic Portrait Animation for AIGC Virtual Humans via Temporally Structured Action Units

Yanshuai Shi
preprint en

Abstract

Affective virtual humans require facial responses that evolve coherently over time, reflect conversational context, and transfer across heterogeneous character appearances. Existing emotional portrait-animation methods commonly rely on discrete emotion labels, frame-wise motion variables, or latent affect representations. This study formulates AIGC-supported virtual-human animation as a context-conditioned temporal affective-behavior modeling problem and proposes TABI-DH. The framework integrates multimodal speech and context encoding, a Temporal AU Behavior Planner, AU-guided non-rigid facial geometry, externally controlled rigid head pose, and behavior–appearance factorized rendering. On MEAD, TABI-DH achieves 22.74 dB PSNR, 0.71 SSIM, 84.25% emotion accuracy, and an AU-trajectory correlation of 0.89. Cross-dataset evaluation on LRW, GRID, and HDTF demonstrates consistent performance under distribution shift. In session-independent IEMOCAP evaluation, incorporating past conversational information increases four-class emotion agreement to 77.1% and reduces affective-response onset error to 176 ms. Mechanism analyses indicate that temporal AU constraints and upper-face behavior primarily improve affective fidelity rather than lip synchronization. Artistic-portrait experiments further show that a common AU trajectory can transfer across photographic and stylized identities while preserving source appearance. These findings support temporally structured AUs as an interpretable representation connecting affective interaction modeling with controllable AIGC virtual-human animation.

Research Square
Zhejiang College of Construction (CN)
Human Motion and Animation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon