SE-TDM: Squeeze-Excitation and Time Differencing Model for Precise Gesture Recognition through Spatio-Temporal Fusion of sEMG and IMU Signals

The integration of sEMG and IMU signals enables accurate gesture recognition, with broad applicability in manipulating prosthetics and human-machine interaction. However, previous fusion methods have predominantly focused on spatial information, often overlooking the crucial temporal information, resulting in inability to adequately portray gestures. This study introduces a novel deep learning method that uses spatio-temporal fusion neural network integrating both sEMG and IMU signals for gesture recognition. In the proposed method, two networks were utilized for the spatial and temporal fusion of sEMG and IMU signals, respectively. The spatial fusion network utilizes the attention mechanisms within the Squeeze-and-Excitation (SE) module to assign varying weights to information. Meanwhile, the temporal fusion network leverages the signal differences across frames, enhancing both short-term and long-term variations to capture the signal temporal information. In order to evaluate the effectiveness of the proposed model, an EMG armband was applied on the wrists of eight subjects to acquire the sEMG and IMU signals. The gesture recognition results indicated that the proposed method achieved a higher gesture recognition accuracy of 93.16±3.08<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">%</sup>, significantly outperformed four state-of-the-art methods (p < 0.05 revealed by ANOVA). This study contributes to offering innovative solutions to MPR in practical applications.