Robustness of Self-Avatar Animation Beyond Sparse Tracking: Effects of Pose Estimator Discrepancies and Inaccuracies

In immersive applications, reconstructing full-body self-avatar motion remains a challenging task, particularly in the absence of lowerbody tracking devices. A promising approach involves supplementing sparse upper-body motion signals with 3D Cartesian joint positions estimated from external RGB cameras. While effective, such pose estimators are prone to inaccuracies, jitter, and occlusions, which can negatively impact reconstruction quality. In this work, we investigate the robustness of self-avatar animation models to such artifacts and to variations across pose estimators. We evaluate two state-of-the-art models, HMD-Poser and AvatarJLM, and show that integrating RGB-based pose data improves lower-body accuracy over VR-only baselines, even for inaccurate 3D pose. However, model performance degrades when training and testing rely on different pose estimators. This highlights the sensitivity of current approaches to estimator variability and underscores the need for estimator-aware training to ensure robustness in real-world deployment.