A Deep Learning-based Static Gesture Categorization and Interpretation of Indian Sign Language (SGCIISLang) Model Using Optimized Gated Recurrent Unit (GRU)

Sign language makes a significant contribution to enabling communication for speech-impaired individuals across the globe. It needs a physical interpreter to facilitate communication; however, these interpreters are limited in number and may not always be available when needed. Deep learning techniques can help address this challenge by creating a virtual interpreter. However, a few notable challenges such as occlusion, external lighting variations, and background subtraction are to be handled in the recognition system. Gestures performed in Indian Sign Language (ISL) involve movements of a single hand or both hands to convey signs for communication. This article proposes a Static Gesture Categorization and Interpretation of Indian Sign Language (SGCIISLang) model that employs an optimized Gated Recurrent Unit (GRU) architecture. For the experiment, we developed our own dataset titled “Static gestures of Indian Sign Language (ISL) for English Alphabet, Hindi Vowels and Numerals” which has already been published on Mendeley Data (https://data.mendeley.com/datasets/7tsw22y96w/1). The MediaPipe library is used for feature extraction, and the results are integrated into our model to classify sign motions. The region of interest is extracted using three MediaPipe approaches: holistic, holistic without pose, and holistic without face. We analyze and compare four models: Long Short Term Memory-Convolutional Neural Network (LSTM-CNN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Gated Recurrent Unit (GRU) based SGCIISLang model. Based on the experimental results, the GRU-based SGCIISLang model achieves greater efficiency, faster processing and rapid convergence compared to CNN, LSTM-CNN and RNN models. The prediction precision achieved is 97.96% with loss, recall, mean square error (MSE) and F1 score values of 0.0837, 0.9742, 2.791 and 0.977, respectively. Our approach outperforms and addresses challenges. The sample prototype of our proposed approach is available on GitHub (https://github.com/AnimeshSingh777/Sample-Prototype-for-Indian-Sign-Language-Static-Gesture-Recognition-System).