Vision-Based Real-Time Gesture-to-Speech Translation for Sign Language

For those who are deaf or hard of hearing, sign language acts as their main means of communication. A gesture-to-speech converter can help close the communication divide between these individuals and the broader community. However, many current approaches come with high computational demands. To address this issue, we propose a convolutional neural network (CNN) architecture optimized for translating sign language gestures to speech, using fewer parameters. Additionally, we integrated the Python library pyttsx3 for text-to-speech conversion. Our method was tested on two publicly available sign language datasets from America and India, achieving accuracy rates of 98.55% and 99.64%, respectively. The proposed approach also outperforms existing techniques in terms of speed. We further validated its effectiveness through a real-time GUI prototype.