Enhancing motor prosthesis training through a gamified mixed-reality system

Abstract During the advanced stages of hand prosthesis training, amputees strive to perfect their movements to enhance fine motor skills, increase speed, and achieve fluid and continuous actions. However, this training typically occurs in abstract, low-engagement scenarios, limited to practicing unilateral hand motions generally without wearing the prosthesis and lacking kinematic performance assessment. We propose a gamified Mixed-Reality (MR) system that allows amputees to practice controlling their actual prosthesis while observing their movements and task performance within a semi-immersive, realistic, and engaging environment. A notable aspect of this system is the ability to facilitate safe practice of grasping maneuvers, supporting both unilateral training and complex bilateral coordination. Ten healthy subjects completed trials involving manipulating virtual objects within an MR environment, establishing ground truth metrics with their collected data. Subsequently, two users with prosthetic hands tested the system, providing initial insights into its applicability for rehabilitation purposes. Performance outcomes and user feedback were analyzed to assess the system’s usability and effectiveness in fostering hand movements and overall user experience. Both participant groups rated the MR training system highly positive; all User Experience Questionnaire (UEQ) factor mean scores (Pragmatic, Hedonic, and Attractiveness) exceeded the $$+1.0$$ threshold. The reference metrics obtained from non-amputee (NA) participants were significant in assessing the quality of movements and performance of amputees in both unilateral and bimanual hand use. Notably, compared to the NA group, prosthetic users exhibited significantly poorer unimanual performance, requiring 90% more time, covering 50% greater distance, and showing 15% reduced smoothness, yet demonstrated equivalent performance during bimanual tasks. Collectively, these findings validate the system’s utility, demonstrating its potential as an innovative and engaging tool for advancing unimanual and bimanual prosthetic training. By providing an opportunity to train with the actual prosthesis, this approach aims to promote more effective skill transfer. The reference metrics generated by NA users, including time, distance, position error, peak velocity, smoothness, and fingertip span, will serve as a benchmark to evaluate prosthesis control and assess the functional progress of amputee users.