A feature-free deep learning approach for midair hand gesture recognition from surface electromyogram (sEMG) data

Midair hand gesture recognition plays a crucial role in applications such as sign language recognition and human-computer interaction, particularly for supporting individuals with partial or complete hearing loss. However, recognizing gestures in midair remains challenging due to the rapid and complex nature of hand movements. To address this, noninvasive techniques like surface electromyography (sEMG)—which captures muscle activity through sensors placed on the skin—have gained attention. sEMG provides rich time-series data that reflect both spatial and temporal muscle dynamics. In this study, we propose a deep learning architecture that combines convolutional neural networks (CNNs) and recurrent neural networks (RNNs) to classify midair hand gestures directly from raw sEMG signals. The CNN extracts spatial features, while the RNN captures temporal dependencies, enabling the model to understand dynamic gestures effectively. An ablation study comparing raw signals, fast Fourier transform (FFT)-based features, and wavelet-based features on two public datasets—Arabic Sign Language (ArSL) EMG and AirScript—demonstrated that raw sEMG signals consistently yielded the highest performance, achieving 94.4% accuracy on the ArSL EMG dataset and 92.7% on AirScript. These results show that our model can accurately classify in-air gestures without the need for additional feature extraction, thereby reducing model complexity and computational overhead. Furthermore, transfer learning was employed to enhance generalization on the smaller AirScript dataset, achieving a boosted accuracy of 95.18%. Overall, the study confirms that direct use of raw sEMG signals with a CNN-RNN model is an effective and efficient solution for midair gesture recognition.