Gesture Recognition Through Mechanomyogram Signals: An Adaptive Framework for Arm Posture Variability

In hand gesture recognition, classifying gestures across multiple arm postures is challenging due to the dynamic nature of muscle fibers and the need to capture muscle activity through electrical connections with the skin. This paper presents a gesture recognition architecture addressing the arm posture challenges using an unsupervised domain adaptation technique and a wearable mechanomyogram (MMG) device that does not require electrical contact with the skin. To deal with the transient characteristics of muscle activities caused by changing arm posture, Continuous Wavelet Transform (CWT) combined with Domain-Adversarial Convolutional Neural Networks (DACNN) were used to extract MMG features and classify hand gestures. DACNN was compared with supervised trained classifiers and shown to achieve consistent improvement in classification accuracies over multiple arm postures. With less than 5 minutes of setup time to record 20 examples per gesture in each arm posture, the developed method achieved an average prediction accuracy of $87.43 \%$ for classifying 5 hand gestures in the same arm posture and $64.29 \%$ across 10 different arm postures. When further expanding the MMG segmentation window from $200 \,\mathrm{ms}$ to $600 \,\mathrm{ms}$ to extract greater discriminatory information at the expense of longer response time, the intra-posture and inter-posture accuracies increased to $92.32 \%$ and $71.75 \%$. The findings demonstrate the capability of the proposed method to improve generalization throughout dynamic changes caused by arm postures during non-laboratory usages and the potential of MMG to be an alternative sensor with comparable performance to the widely used electromyogram (EMG) gesture recognition systems.