Sign Language Recognition and Translation Using Self-Attention Long-Short-Term Memory with Shape Autotuning Activation Function

An efficient Indian Sign Language recognition (ISL) identifies the sign language gestures to ease communication among the non-signer and signer community. It is a graphic form of communication where the community of hearing-impaired exists and this sign language raises independently from local spoken language. However, sign language recognition and translation face constraints due to variability in sign production between individuals and contextual ambiguity which makes it difficulties for models to generalize and accurately interpret signs in various contexts. This research proposes Self-Attention Long-Short-Term Memory with Shape Autotuning Activation Function (SALSTM-SAAF) to recognize and translate sign language. RA allows the model to focus on essential parts and LSTM captures temporal dependencies which ensures effective sequence learning. SAAF adjusts to different input shapes which optimizes the learning process and enhances the model’s performance. Data augmentation and min-max normalization are applied to increase the dataset size and normalize the obtained data. Then, the ResNet is used to extract the features from pre-processed data effectively. The SALSTM-SAAF achieves a better accuracy of 99.87% compared to existing methods like VGG19-Bidirectional LSTM (VGG19-BiLSTM).