Spatial Navigation Concepts Based on Pose Estimation for a VR-CAVE Setting

This paper presents a technical proof-of-concept for utilizing cost-effective, computer vision-based pose estimation to enable natural and virtual navigation in VR- CAVEs. Utilizing MediaPipe for pose estimation we examined key challenges in applying pose data to navigation concepts. To address issues such as low transmission rate of pose data, joint position fluctuations, and difficulties in capturing fast movements, we implemented a processing pipeline that normalizes extremity lengths, filters pose data with a Kalman filter and classifies hand gestures using minimal training data. We developed a prototype (LCVR-CAVE) to test natural navigation through user-centred perspective adjustments based on filtered head positions, and virtual navigation concepts using gesture-based controls and 3D arm raycasts. The virtual navigation methods include teleportation, panning, flying, and view rotation, activated through specific hand gestures and movement patterns. An initial evaluation indicated that while natural navigation provided an immersive experience, the effectiveness of virtual navigation concepts varied depending on the implemented navigation approach, with low pose sampling rates having the most significant impact.