ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer

We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech. The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture to explicitly model co-speech motion dynamics, enabling joint spatial-temporal dependency modeling. We enhance the model’s robustness, noise resistance and cross-domain generalization. To mitigate inherent limitations of autoregressive inference, including error accumulation and limited self-correction, we propose an iterative reconstruction inference (IRI) strategy, which refines motion sequences via cyclic pose reconstruction. Extensive experiments on benchmark datasets validate ReCoM’s effectiveness, achieving state-of-the-art performance across metrics. Notably, it reduces the Fréchet Gesture Distance (FGD) from 18.70 to 2.48, demonstrating an 86.7% improvement in motion realism.