Hand Gesture Recognition for Indian Sign Language: A CNN-GRU-MLP Approach

Addressing communication barriers faced by the hearing-impaired through technology not only promotes inclusivity but also enriches social interaction and fosters equal opportunities.The proposed model to recognize hand gesture in Indian Sign Language (ISL) utilizes a fusion of convolutional neural network (CNN), multi-layer perceptron (MLP) architectures, and gated recurrent unit (GRU).The dataset for this study is accessed from the Kaggle repository, with pre-processing and augmentations applied to enhance data quality and diversity, ensuring robust model performance.The CNN module serves as a generalization model, identifying common features across different gestures through spatial pattern recognition, while the MLP component further refines these features, creating a dense feature representation.The GRU module functions as a memory model, learning temporal and sequential dependencies in individual gestures by categorizing features into dense and continuous types.The suggested system was trained on a large dataset of 35 distinct hand gestures to achieve high accuracy in identifying and differentiating between various gesture classes.The training process was optimized over 100 epochs, during which the model demonstrated a steady improvement in training and validation accuracy.The proposed model achieved a high accuracy of 96.82%, with precision, recall, and F1-score values of 96.75%, 96.01%, and 96.37%, respectively.This highlights the model's effectiveness in accurately recognizing 35 distinct hand gestures in Indian Sign Language.The training and validation loss also consistently decreased, confirming the model's capacity for effective pattern learning without overfitting.The suggested system can be effectively utilized in various domains requiring accurate hand gesture recognition, such as virtual reality, human-computer interaction, and assistive technology for individuals with disabilities.This work contributes to the field by providing a robust and scalable solution for gesture-based control systems, demonstrating the potential for enhancing interaction experiences through reliable and precise gesture recognition.