Generation of Avatar’s Full-Body Locomotion from Head and Hand Movements Using Motion Matching

In a cross-reality environment using HMDs (Head Mounted Devices), reflecting the user’s whole-body posture in real-time on the avatar is important for immersive experiences. However, the motion information that can be obtained from a set of HMD devices is limited to the head and hands. There are existing methods that utilize deep learning to generate postures from limited motion information, but they cause large delays in the system. In this study, we present a method to generate whole-body motions of an avatar with low latency using motion information only from the HMD and left and right controllers. We designed new features to extract full-body motions corresponding to head and hand motions and developed a system to generate walking motions of avatars from a small data set with a small delay by motion matching. We present experimental results and evaluate the effectiveness of the proposed method.