Class-Conditional Human Motion Generation using StyleGAN and Video Classifier

In movies and games utilizing 3D computer graphics, humanoid characters are often expected to appear and behave like real humans. This study aimed to develop a system for generating and controlling these Computer Graphics characters by specifying motion classes. We first employ the StyleGAN architecture to learn from motion data captured using a motion-capture system, enabling the generation of diverse and natural human movements. Next, the motion data randomly generated by this model are converted into video sequences through differentiable rendering. These sequences are then input into a motion-video classifier to predict the probability of each class. Finally, the latent variables of the motion-generation model are iteratively updated, guiding it toward generating motions corresponding to the desired class. Experimental results confirmed the effectiveness of the proposed method in generating appropriate motions across various motion classes.