Virtual reality rehabilitation using hand tracking: interaction system design and usability tests

Abstract Virtual reality (VR) rehabilitation can complement traditional therapy by enhancing motivation and exercise quality, while effectively supporting the transfer of skills to real-world tasks. We present a VR interaction system based on hand tracking, developed by a multidisciplinary team of clinicians and engineers, designed to support a variety of hand interactions. The system features adjustable task difficulty and requirements, performance metrics, visual error augmentation (EA), and audio-visual signifiers to guide user interactions. A proof-of-concept rehabilitation game was tested with non-expert (N = 7) and expert occupational therapists (N = 5). Participants completed tasks including grabbing objects, pinching, pulling handles, tracing paths, and making hand gestures. Usability was the primary study outcome, with results indicating high usability scores: a modified System Usability Scale (SUS) of 88.9 for non-experts and 74.5 for experts. Expert participants unanimously recognised the system’s potential for rehabilitation and identified multiple clinical populations that could benefit from its functionalities. However, findings also highlight the need to improve the communication of virtual object affordances, which may influence user engagement and task performance.