Real-Time Supervision and Guidance for Rehabilitation Exercises via Computer Vision

Rehabilitation exercises are essential for individuals who have lost their ability to function normally. Typically, a functional rehabilitation program consists of two parts: supervised exercises in a rehabilitation center and additional exercises performed independently at home. Home exercises are as important as in-center exercises and can significantly improve recovery time. However, at home, the patient is unsupervised, with a significant risk of performing exercises incorrectly. This article presents a web application designed for hand rehabilitation exercises that uses a pretrained hand landmark model and an artificial neural network (ANN) classifier to ensure correct exercise performance. The application employs transfer learning to extract features from the user's hand movements, with the ANN classifier, designed using the ML5 library, determining if the positions are performed accurately. The ML5 library, a high-level interface to TensorFlow.js, makes the application suitable for web deployment. Physicians can create new exercises and retrain the ANN classifier without programming skills. An evaluation with 12 participants demonstrated an overall classification accuracy of 90.25% and achieved a system usability scale score of 75.30, indicating high usability and potential for real-world rehabilitation scenarios. This lightweight web application is highly accessible and user friendly and runs seamlessly on any smartphone or tablet. This approach shows great promise in enhancing exercise accuracy and safety, thereby improving the rehabilitation process at home and in clinical settings. In addition, it is adaptable for other models for full-body and face tracking, making it versatile for various rehabilitation applications.