SE-Former: Skeleton-Enhanced Learning Framework for 3D Human Motion Prediction

The key challenge in 3D human motion prediction lies in the inaccurate modeling of spatial and temporal dependencies in motion sequences. The latent feature relationships within the data remain underexplored, particularly with incomplete data (missing joints). Motivated by the coordinated motion of multiple skeletons in the human body, this paper presents SE-Former, a skeleton-enhanced Transformer model designed to advance the feature representation of human motion. It leverages the spatial-temporal information of motion sequences, and mines the latent skeletal spatial information to adapt to the intrinsic properties of the data. Specifically, SE-Former introduces virtual joints with noise and simulates realistic joint combinations, which are then integrated with native skeletal features via a graph convolutional network-attention encoder. Considering the long-term prediction is easily hindered by the model tendency to converge to average poses, we draw inspiration from trend modelling techniques in the biomechanics domain. A trend learning module is integrated to capture long-term motion trends and residual dynamics. It effectively bridges the encoder and decoder, ensuring that predictions maintain dynamic variability, so as to avoid the convergence toward average poses and improve the fidelity of motion predictions. Extensive experiments are conducted on Human3.6M, CMU-MoCap and 3DPW datasets, where our SE-Former shows significant advantages over the other state-of-the-art work. Code is available at https://github.com/Logan-007L/SEFormer .