Posture Estimation Model Combined With Machine Learning Estimates the Radial Abduction Angle of the Thumb With High Accuracy

The thumb function is complex, and accurate evaluation through images or videos is difficult. Pose estimation, a technology that uses artificial intelligence (AI) to estimate skeletal detection of the body, is gaining popularity. In this study, we combined the pose estimation library MediaPipe-Hands and five machine learning (ML) models to predict the radial abduction angle of the thumb. Radial abduction movements of 20 hands from 10 healthy volunteers were captured on video and processed into 5,000 images. Angle measurements by goniometer were used as true values to evaluate the angle reliability of the MediaPipe-Hands and the angle reliability of the MediaPipe-Hands combined with ML. The correlation coefficient (CC) between the angle measured by goniometry and the angle calculated by MediaPipe-Hands was 0.84. In contrast, applying ML to MediaPipe-Hands resulted in models with improved accuracy, and all models showed high CCs (0.94-099) with angle measurements taken by goniometry. The ML model also predicted the abduction angles when the camera was taken from three different angles. In visualizing the features that the AI deemed important, the ML model predicted the abduction angle by focusing on the tip distance between the thumb and index finger along with the angle of the metacarpophalangeal joint between the thumb and middle finger. These results enable angle estimation even without frontal imaging with a camera, and expansion of this system may lead to real-time functional assessment in telemedicine and rehabilitation without the need for physical contact.