Vision-Based Human–Computer Interaction System for Mexican Sign Language Recognition and Robotic Reproduction: Toward Inclusive Educational Applications

Mexican Sign Language (MSL) plays an essential role in communication within the Deaf community, yet access to sign-language learning resources remains limited in some educational settings. This study presents a vision-based Human–Computer Interaction (HCI) system for MSL recognition and robotic reproduction, developed as a proof of concept toward inclusive educational applications. The system uses an RGB camera and MediaPipe Hands for hand-landmark extraction, a multilayer perceptron (MLP) for recognition of 29 static classes, and a TimeDistributed–Long Short-Term Memory (LSTM) model for 10 dynamic classes. Recognized signs are mapped to predefined motion sequences executed by an ATmega2560-controlled robotic arm. Under stratified random and sequential partitioning protocols, the static model achieved accuracies of 100.00% and 99.88%, respectively, while the dynamic model achieved 100.00% under both protocols. Additional comparisons with Support Vector Machine (SVM) and Random Forest classifiers, using three random seeds, demonstrated competitive performance and low variability within the original single-participant dataset. To assess generalization beyond the training participant, a preliminary signer-independent evaluation was conducted with three previously unseen participants, without model retraining or adaptation. Across 585 recognition attempts involving 39 classes, the system achieved an overall accuracy of 89.06%, providing initial evidence of recognition performance across different signers. Furthermore, 92 accepted recognition trials successfully triggered their corresponding robotic sequences, demonstrating functional integration between recognition and actuation. These findings support the technical feasibility of the proposed contactless HCI architecture as a potential foundation for interactive MSL learning tools. Nevertheless, broader signer-independent validation and user-centered evaluations are required to establish its accessibility, usability, and educational effectiveness.