Smartphone-based real-time feedback to suppress trunk compensation for unsupervised upper limb rehabilitation in patients with brain disorders

BACKGROUND: To ensure continuous rehabilitation for patients with hemiparesis due to brain disorders, there is a growing need for simple, accessible systems that enable unsupervised self-training. The occurrence of compensatory trunk movements during the unsupervised exercise may prevent true functional recovery. This study proposes a smartphone-based visual-auditory feedback system designed to detect and suppress compensatory trunk movements in real-time without wearable sensors. METHODS: A pilot cohort (n = 16) was first used to calibrate detection thresholds for compensatory trunk movements. Subsequently, a total of 55 hemiparetic patients were enrolled in a randomized controlled trial and allocated to a Feedback (FB) group (n = 27) or a Non-Feedback (NFB) group (n = 28). Participants performed standardised upper limb rehabilitation tasks using the Rapael Smart Board™, a planar upper limb rehabilitation device. The proposed system utilised a smartphone camera with MediaPipe-based pose estimation to track trunk movements and provided real-time traffic-light feedback based on calibrated thresholds. Outcome measures included trunk path length, trunk deviation, task efficiency, and spatial occupancy, which were evaluated using 3D coordinates reconstructed from depth camera data. RESULTS: The FB group demonstrated significantly improved postural stability compared to the NFB group, with a 37.9% reduction in trunk path length (p = 0.014) and a 35.3% decrease in spatial occupancy (p = 0.003). Kinematic analysis revealed that the NFB group's shorter hand path lengths were achieved through kinematic redundancy-specifically, the recruitment of trunk degrees of freedom-rather than through selective upper limb motor control. In contrast, the FB group maintained a stable posture near the neutral position, ensuring true upper limb engagement. The usability assessment demonstrated that the system was well received by users and was reliable (Cronbach's α= 0.775). CONCLUSION: The proposed system, which integrates mobile technology, effectively suppresses compensatory trunk movements and promotes selective motor control, and ensures that rehabilitation outcomes reflect true upper limb joint engagement rather than kinematic redundancy through compensatory trunk recruitment. While certain design considerations remain, particularly related to dynamic recalibration and the use of a fixed auditory feedback window, the system retains strong potential as an automated feedback solution, offering a scalable and accessible pathway for high-quality unsupervised home rehabilitation.