IMU-weighted multimodal fusion for robust gesture recognition across arm positions

OBJECTIVE: Human-machine interfaces (HMIs) based on surface electromyography (sEMG) and A-mode ultrasound have achieved high gesture recognition accuracy; however, most studies have been conducted under a single static arm position. In practice, arm position variation degrades performance and hinders deployment in neural prostheses. Although Amode ultrasound can outperform sEMG at fixed positions, it is more sensitive to positional changes, an issue that has rarely been examined. This study addresses the cross-position sensitivity of ultrasound-based HMIs through multimodal fusion with sEMG and an inertial measurement unit (IMU). APPROACH: We propose an IMU-weighted multimodal fusion framework. A feature fusion model and an sEMG single-modality model are trained in parallel. At the prediction stage, the arm position is estimated from the IMU, and the angular offset relative to the training positions is used to compute a continuous weight for combining the posterior probabilities of the two models. Near training positions, greater weight is placed on the high-accuracy fusion model; as the offset grows, the contribution progressively shifts toward the robust sEMG model to avoid further degradation caused by A-mode ultrasound. MAIN RESULTS: Experiments were conducted across nine continuously varying arm positions from forearm elevation to lowering. With only two training positions, the proposed IMUweighted method achieved an average accuracy of 85.88% across all nine positions, significantly outperforming other models. SIGNIFICANCE: The method effectively exploits the high static accuracy of A-mode ultrasound and the cross-position stability of sEMG. With a low training burden, the proposed method raises the lower bound of cross-arm-position decoding accuracy, providing a practical route toward robust A-mode-ultrasound-based prosthetic control across varying arm positions.