A Novel Rehabilitation Training System for Hand Amputation Based on Mixed Reality

Hand loss impacts physical, mental, and social aspects of individuals' lives. Prosthetics offer a crucial pathway to regaining lost functionality. However, challenges such as phantom limb pain, discomfort, and high abandonment rates persist. This study introduces a rehabilitation system that combines mixed reality with prosthetic hand. The virtual interface includes four buttons, each triggered by a 3-second gaze: start recognizing, display virtual objects, start grasping, and exit. Start recognizing button activates object recognition, which outputs the corresponding pre-grasp gesture to guide the user. After starting grasping, the user will see both the virtual object and a virtual hand attached to the real prosthetic hand. The enhanced visual feedback is designed to improve the user's control ability over the prosthetic. Results showed that with the help of visual feedback, adjustment of grasp force is about 1s shorter than the situation without visual feedback. In the experiment where each object was grasped 30 times, the success rate of grasping the ball was 96.67%, the success rate of grasping the cylinder was 100%, and the success rate of grasping the cube was 90%. By integrating mixed reality interface and prosthetic hand, the system aims to accelerate the response time of prosthetic hand grasping and reduce cognitive load during rehabilitation training. The system might support amputees in performing more personalized rehabilitation tasks, ultimately facilitating early and successful adaptation to prosthetics.