Mutual Information Regularization for Cross-Subject Gesture Recognition Using Surface Electromyography and Acceleration

Feature fusion enables joint classification of surface electromyography (sEMG) and acceleration (ACC) signals, but classification supervision alone does not explicitly optimize statistical dependence between their learned representations. To address this gap, we propose a dual-branch gesture recognition method that combines supervised classification with mutual information (MI) regularization. A Mutual Information Neural Estimation (MINE)-based dependence regularizer encourages cross-modal dependence during training; the auxiliary projection and critic are removed at inference. The experiments included 11 participants performing 10 gestures under four postures across five repetitions, with within-subject testing and strict leave-one-subject-out evaluation. The method achieved an accuracy/Macro-F1 of 93.78%/93.74% within subjects, 37.02%/34.72% before cross-subject adaptation, and 85.61%/85.57% after adaptation using one target repetition of 40 gesture–posture trials (9 min 20 s of scheduled recording). Compared with the Basic sEMG–ACC fusion baseline using the same recognition architecture without MI regularization, accuracy increased by 2.52, 6.00, and 3.06 percentage points in the three settings, respectively; all matched comparisons remained significant after Holm correction. After one-repetition adaptation, MI also outperformed the evaluated local implementations of MSCNN-TL and an ACC-adapted Multistream CNN with fine-tuning. MI achieved higher mean performance than cross-temporal attention alone (CTA-only) and its combination with MI (CTA+MI), although the cross-subject differences were not significant after Holm correction. These results support MI regularization as an effective addition to classification supervision in the evaluated cross-subject sEMG–ACC framework. The current evaluation focuses on offline steady-action classification and does not cover continuous interaction scenarios such as gesture detection and transition handling.