Harnessing attention-driven hybrid deep learning with combined feature representation for precise sign language recognition to aid deaf and speech-impaired people

Speech is the primary form of communication; still, there are people whose hearing or speaking skills are disabled. Communication offers an essential hurdle for people with such an impairment. Sign Languages (SLs) are the natural languages of the Deaf and their primary means of communication. As visual languages, they use numerous corresponding channels to transfer information. This includes manual features, such as hand pose, shape, and movement, as well as non-manual features, including mouth movements, head, shoulder, torso, and facial expressions. SL recognition (SLR) consists of the complete procedure of following and recognizing the signs achieved and transforming them into semantically essential words. SLR is a visual language which communicates meaning through body and hand gestures. Currently, much research work in SLR, depending on the deep learning (DL) model, is implemented on SLs. This paper proposes an Attention-Driven Hybrid Deep Learning Model with Feature Fusion for Accurate Sign Language Recognition (AHDLMFF-ASLR) model. The primary goal of the AHDLMFF-ASLR model is to enhance SLR for deaf and mute individuals by utilizing advanced techniques for accurate, real-time gesture recognition. In its initial stage of image pre-processing, contrast-limited adaptive histogram equalization (CLAHE) is used to enhance image details, and Canny edge detection (CED) is employed to emphasize the edges of objects. Furthermore, the feature extraction process integrates the Swin Transformer (ST), ConvNeXt-Large, and ResNet50 models. Finally, the AHDLMFF-ASLR model utilizes a hybrid of a convolutional neural network and bidirectional long short-term memory with attention (C-BiL-A) technique for the classification process. The efficiency of the AHDLMFF-ASLR technique is examined under the SL dataset. The comparison study of the AHDLMFF-ASLR technique revealed a superior accuracy value of 98.10% compared to existing models.