Multi-Task Driven Adapter-Based Foundation Model for Locomotion Prediction in Virtual Reality

Serving as a fundamental interaction in Virtual Reality, Locomotion technology defines how users navigate and explore the immersive virtual environments with high degree of freedom. Accurate prediction of locomotion not only enhances the sense of presence and ease of movement in virtual environment but also benefits VR applications through context-aware optimization such as pre-rendering and scene streaming. To leverage the superior understanding and causal modeling capabilities of Foundation Models (FM) in the domain of numerical prediction, we apply FMs to time-series data, enabling more accurate estimation of users’ future spatial coordinates based on historical motion and gaze data. In this paper, we introduce LoCoFoMo, an Adapter-based Foundation Model architecture specifically designed for future trajectory prediction in VR contexts. We conduct extensive experiments to evaluate the effectiveness of LoCoFoMo and its components. The proposed model demonstrates strong competitiveness when compared with several trajectory prediction and temporal reasoning models, evidenced by a performance gain of over 25% against the best baseline on datasets with interaction paradigms like Touchpad and Arm-Swing, coupled with superior stability in mitigating error accumulation.