PropGesture: Few‐shot Co‐speech Gesture Synthesis for Handheld Props

Abstract While co‐speech gesture synthesis has made significant progress in generating natural full‐body motion from speech, a frequent setting where characters hold objects while speaking remains largely underexplored. The core challenge is that motion capture data cannot be collected at scale for every possible object, making few‐shot adaptation a practical necessity. Yet a handful of samples is hard to exploit: conditioning on them as motion examples imposes only a soft prior, yielding intermittent object‐aware behavior, while finetuning the full model on them either overfits and loses gesture diversity or fails to produce stable, prop‐appropriate motion on the holding arm. To address this, we present PropGesture, a few‐shot framework that adapts to a target prop using a small set of example clips synthesized by a video generation model. Our key design is to localize object adaptation to the prop‐holding arm, preserving gesture diversity while introducing object‐specific dynamics locally. Specifically, a phase‐based arm motion generator, pretrained on large‐scale gesture data and finetuned on the example clips, resolves the many‐to‐many speech–gesture ambiguity through phase sampling and retrieves arm motion via motion matching. This arm motion is then inpainted into a pretrained gesture flow matching model that synthesizes the remaining body from speech. We demonstrate the effectiveness of our method through both perceptual and quantitative comparisons against motion‐example‐based and finetuning‐based baselines. Source code and dataset are available at our project page.

Authors

Institutions

Publication Details

Journal
Computer Graphics Forum
Published
2026-11-13
DOI
https://doi.org/10.1111/cgf.70603
Primary Topic
Human Motion and Animation
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

PropGesture: Few‐shot Co‐speech Gesture Synthesis for Handheld Props

Sung‐Hee Lee, J.H. Kang, Jihun Shin, Junghoon Choi
3 links
Computer Graphics Forum
Human Motion and Animation
article

PropGesture: Few‐shot Co‐speech Gesture Synthesis for Handheld Props

Sung‐Hee Lee, J.H. Kang, Jihun Shin, Junghoon Choi
article en

Abstract

Abstract While co‐speech gesture synthesis has made significant progress in generating natural full‐body motion from speech, a frequent setting where characters hold objects while speaking remains largely underexplored. The core challenge is that motion capture data cannot be collected at scale for every possible object, making few‐shot adaptation a practical necessity. Yet a handful of samples is hard to exploit: conditioning on them as motion examples imposes only a soft prior, yielding intermittent object‐aware behavior, while finetuning the full model on them either overfits and loses gesture diversity or fails to produce stable, prop‐appropriate motion on the holding arm. To address this, we present PropGesture, a few‐shot framework that adapts to a target prop using a small set of example clips synthesized by a video generation model. Our key design is to localize object adaptation to the prop‐holding arm, preserving gesture diversity while introducing object‐specific dynamics locally. Specifically, a phase‐based arm motion generator, pretrained on large‐scale gesture data and finetuned on the example clips, resolves the many‐to‐many speech–gesture ambiguity through phase sampling and retrieves arm motion via motion matching. This arm motion is then inpainted into a pretrained gesture flow matching model that synthesizes the remaining body from speech. We demonstrate the effectiveness of our method through both perceptual and quantitative comparisons against motion‐example‐based and finetuning‐based baselines. Source code and dataset are available at our project page.

Computer Graphics Forum
Korea Advanced Institute of Science and Technology (KR)
Ministry of Science and ICT, South Korea
Human Motion and Animation
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon