Enhanced Control of Human Motion Generation using Action-conditioned Transformer VAE with Low-rank Factorization

This paper presents an action-conditioned transformer variational autoencoder (VAE) designed to generate realistic and diverse human motion sequences. The model enables control of specific body parts of the generated human motions, thereby achieving more degrees of freedom and diversity in human actions. In order to achieve control of the body parts, this paper acquires attribute vectors through low-rank factorization and null space projection. We employ scheduling schemes for the KL-term ( β ) and data augmentation to address posterior collapse to promote motion diversity. Evaluations on the UESTC and HumanAct12 datasets demonstrate the effectiveness of the proposed model and methods, showing plausible and humanlike actions. In addition, we show the application of control to actions generated in unconditional settings, thus revealing the potential for future research. To the best of our knowledge, this is a pioneering work on directly controlling motions in the latent space without using other modalities.