Hand Sign Translator Bridging the Communication Gap for ISL Users

Millions of individuals with hearing disabilities, or who are deaf, in India experience significant barriers when communicating in-person with individuals who do not know Indian Sign Language (ISL). This barrier often restricts their access to or utilization of vital services, education, and social interactions. The Hand Sign Translator is a real-time solution that reliably recognizes ISL hand gestures to improve communication within these contexts. Built with Python primarily, the Translator consists of OpenCV and deep learning techniques, assuring reliable hand detection and feature extraction with MediaPipe. The hand gestures are converted to text using a hybrid model consisting of Convolutional Neural Networks (CNN), and a Bidirectional Long Short-Term Memory (BiLSTM). The model trained on ISL data frequency from various users, providing the model with the ability to generalize to various signing styles while reliably giving consistent output. The Hand Sign Translator decreases the communication barrier between the deaf community and the general public by recognizing hand gestures, and translating it into written text. The application also enhances social and economic inclusion of individuals with hearing disabilities by developing a new and inclusive assistive technology.