Analysis and improvement of surface electromyography-based American Sign Language hand gesture recognition

Surface electromyography (sEMG) allows sign language gestures to be recognized from forearm muscle activity without visual access to the hand. Gestures that produce similar forearm activation patterns, however, remain difficult to distinguish. This study examined how these similarities relate to classification errors and whether short-term temporal context improves recognition of static American Sign Language (ASL) gestures. We recorded 32-channel forearm sEMG from 10 participants performing 24 static gestures from the ASL alphabet and evaluated a dual-branch convolutional-Transformer classifier using both participant-specific and shared multi-user models. We also applied a Multi-Window Model that integrated the current and preceding window representations through causal attention after offline zero-phase filtering. On identical target windows, single-window classification achieved letter-only accuracies of 86.71% and 85.23% in the two settings, respectively. Nine low-recall letters formed three signal-similar groups, {A, M, N, S, T}, {E, O}, and {U, V}, with classification errors strongly concentrated within these groups. Spatial activation and independent-component analyses further showed that greater similarity in forearm sEMG patterns was associated with greater classification confusion. With the Multi-Window Model, letter-only accuracy increased to 90.77% in the participant-specific setting and 89.01% in the multi-user setting, with improvements observed across all participants in the participant-specific setting and larger gains for the nine low-recall letters. These findings demonstrate that integrating recent signal history can improve the accuracy of sEMG-based sign language gesture recognition, while persistent errors among highly similar gestures indicate an important limitation of the information available from forearm sEMG alone.