Spatiotemporal motion modeling with VAE-GNN framework in optical multimodal perception scenarios

To address the insufficient modeling of complex spatiotemporal dynamics in dance motion prediction, this paper proposes DanceAST-VAE, a variational autoencoder-based model integrated with optical motion capture technology. Leveraging a multimodal sensing architecture in the context of smart education, the model decomposes dance motion sequences into diffusion signals, inherent signals, and random signals through variational inference and residual degradation mechanisms. To effectively capture multiscale spatiotemporal dependencies, we design heterogeneous learning modules: graph-based spatiotemporal convolutions model dancer interactions in diffusion signals, while temporal attention mechanisms extract rhythmic and stylistic patterns from inherent signals. An optical multispectral fusion strategy is employed to enhance cross-channel feature integration and ensure robust motion representation. Experiments conducted on optical motion datasets demonstrate that DanceAST-VAE outperforms state-of-the-art methods by 15.2% in prediction accuracy, while also achieving superior adaptability in improvisational scenarios. These results validate the model’s capability to generalize across diverse motion patterns and performer identities. This study provides novel technical support for intelligent choreography assistance systems and immersive VR-based dance education, contributing to the development of personalized teaching and learning strategies within the smart education paradigm.