Gesture-based language detection for people with disabilities using AI technology

The Translation System (TS) is an essential tool for individuals with disabilities, helping them communicate more effectively with society. With the increasing role of computers in daily life, advancements in human-computer interaction have greatly benefitted mute and deaf communities. Sign language plays a crucial role in bridging communication gaps for those with hearing impairments. Our proposed system is an intelligent translator designed to convert sign language gestures into text, facilitating communication between people with speech impairments and the wider public. It offers a reliable and efficient solution for both individuals with speech challenges and those without. The system uses the pre-trained NASNetLarge model, trained on a Sign Language dataset with 27,455 training samples and 7,172 testing samples. The model was evaluated after training using several performance metrics, such as accuracy rate, precision, recall, and F1-score, as well as the confusion matrix. Additionally, the class matrix was displayed. The model achieves a 96% accuracy rate, demonstrating its effectiveness in converting sign language into text.