Synergy-Based Evaluation of Hand Motor Function in Object Handling Using Virtual and Mixed Realities

This study introduces a novel system for evaluating hand motor function through synergy-based analysis during object manipulation in virtual and mixed-reality environments. Conventional assessments of hand function are often subjective, relying on visual observation by therapists or patient-reported outcomes. To address these limitations, we developed a system that utilizes the leap motion controller (LMC) to capture finger motion data without the constraints of glove-type devices. Spatial synergies were extracted using principal component analysis (PCA) and Varimax rotation, providing insights into finger motor coordination with the sparse decomposition. Additionally, we incorporated the HoloLens 2 to create a mixed-reality object manipulation task that enhances spatial awareness for the user, improving natural interaction with virtual objects. Our results demonstrate that synergy-based analysis allows for the systematic detection of hand movement abnormalities that are not captured through traditional task performance metrics. This system demonstrates promise in advancing rehabilitation by enabling more objective and detailed evaluations of finger motor function, facilitating personalized therapy, and potentially contributing to the early detection of motor impairments in the future.