PropGesture: Few‐shot Co‐speech Gesture Synthesis for Handheld Props
Abstract While co‐speech gesture synthesis has made significant progress in generating natural full‐body motion from speech, a frequent setting where characters hold objects while speaking remains largely underexplored. The core challenge is that motion capture data cannot be collected at scale for every possible object, making few‐shot adaptation a practical necessity. Yet a handful of samples is hard to exploit: conditioning on them as motion examples imposes only a soft prior, yielding intermittent object‐aware behavior, while finetuning the full model on them either overfits and loses gesture diversity or fails to produce stable, prop‐appropriate motion on the holding arm. To address this, we present PropGesture, a few‐shot framework that adapts to a target prop using a small set of example clips synthesized by a video generation model. Our key design is to localize object adaptation to the prop‐holding arm, preserving gesture diversity while introducing object‐specific dynamics locally. Specifically, a phase‐based arm motion generator, pretrained on large‐scale gesture data and finetuned on the example clips, resolves the many‐to‐many speech–gesture ambiguity through phase sampling and retrieves arm motion via motion matching. This arm motion is then inpainted into a pretrained gesture flow matching model that synthesizes the remaining body from speech. We demonstrate the effectiveness of our method through both perceptual and quantitative comparisons against motion‐example‐based and finetuning‐based baselines. Source code and dataset are available at our project page.
Authors
- Sung‐Hee Lee (ORCID: https://orcid.org/0000-0001-6604-4709)
- J.H. Kang (ORCID: https://orcid.org/0000-0001-9766-939X)
- Jihun Shin (ORCID: https://orcid.org/0009-0001-9758-4983)
- Junghoon Choi (ORCID: https://orcid.org/0009-0007-7962-7509)
Institutions
- Korea Advanced Institute of Science and Technology (KR)
Publication Details
- Journal
- Computer Graphics Forum
- Published
- 2026-11-13
- DOI
- https://doi.org/10.1111/cgf.70603
- Primary Topic
- Human Motion and Animation
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- Ministry of Science and ICT, South Korea