Deep-SignSpeak: Deep Learning based Sign Language Recognition and Regional Language Translation

Hearing impaired and silent people have a tough time communicating with others in society. It may undermine their self-esteem and make them feel separated from others. Traditional Hand Gesture Recognition Systems face limitations as they heavily rely on explicit hand-crafted features, hindering adaptability to diverse signing styles. Their ability to capture intricate spatial and temporal characteristics is diminished, and their scalability is affected. There are issues with managing changes in ambient light and noise level. Rule-Based Systems, which are based on pre-established rules, are limited in their ability to adjust to the dynamic nature of sign languages, are unable to handle the complexities of different types of signing, and have a tendency to oversimplify sign language patterns. The accuracies of these existing systems were not that much compromising, so that in order to produce an optimum model for sign language interpretation deep learning models are used. This proposed system offers a thorough approach to Sign Language Recognition(SLR).This system focuses on breaking down video input into individual image frames and building three different models: EfficientNetV2, EfficientNetV2L, and ConvNeXtLarge algorithm. The dataset used includes a variety of sign languages, offering a representative and varied sample for training the model. Feeding the dataset into the models during the training phase helps the models become proficient in identifying and deciphering sign language motions. The model's effectiveness is then thoroughly tested by dissecting each individual frame that is produced from videos and evaluating performance measures. After the top-performing model is chosen, there is a transformative stage in which a translator is used to translate the model's output which is predicted is now converted into 22 different regional languages based on the user’s need. The accuracy yielded by the three models EfficientNet_V2, EfficientNet_V2L and ConvNextLarge are 94.20%, 92.54% and 95.21% respectively. The model with the highest accuracy is preferred for recognizing the Sign Languages.