Multimodal Hands-Free VR For Wheelchair Users With Upper Limb Mobility Limitations: Leaning, Head-Gain, and Gaze Pointing

The immersive experiences and social connectivity offered by Virtual Reality (VR) make it a powerful tool for promoting inclusion and accessibility. However, many VR games and applications frequently rely on body tracking or bi-manual controllers, excluding users with upper limb mobility limitations, especially wheelchair users. For these users, simultaneously managing handheld controls and manual wheelchair navigation represents a challenge and reduces engagement and the perception of accessibility. To address this, we propose a multimodal hands-free system designed for wheelchair users with upper limb mobility restrictions. Our system uses HMD-integrated sensors to perform navigation and pointing (selection) tasks in virtual environments. It combines a leaning-based locomotion system, dynamic head-gain rotation, and gaze-based pointing for precise interaction. This solution eliminates the need for additional equipment and accommodates the physical demands of these users. This paper outlines the design principles and implementation of the multimodal system, aiming to advance accessibility in VR for wheelchair users.