Dancing Points: Synthesizing Ballroom Dancing With Three‐Point Inputs

Abstract Ballroom dancing is a structured yet expressive motion category. Its highly diverse movement and complex interactions between leader and follower dancers make the understanding and synthesis challenging. We demonstrate that the three‐point trajectory available from a virtual reality (VR) device can effectively serve as a dancer's motion descriptor, simplifying the modelling and synthesis of interplay between dancers' full‐body motions down to sparse trajectories. Building on this observation, our central design decomposes dual‐character synthesis into a non‐autoregressive mapping network that predicts the future three‐point trajectories of both dancers from the leader's input, followed by two independent full‐body tracking problems. Disentangling the dual‐character interaction from the full‐body autoregressive rollout is what enables stable and responsive real‐time generation on a small dataset, and avoids the overfitting that arises when conditioning one autoregressive model on another. The same decomposition further reduces the per‐step ambiguity of three‐point tracking, so the tracker can be simplified to a lightweight deterministic model despite the task being typically considered under‐constrained and addressed with generative models. We further show that the design generalizes beyond small, structured ballroom dancing datasets and performs robustly on larger, more diverse datasets such as LaFAN [HYNP20]. Our method provides a computationally‐ and data‐efficient solution, opening new possibilities for immersive paired dancing applications.