Sign2Speech: A Novel Approach to Real Time Sign Language to Speech Conversion with Convolutional Neural Networks

To mitigate the communication barrier between hard-of-hearing individuals and individuals who lack American-Sign-Language (ASL) knowledge, a real time ASL to speech converter project was developed. A pre-trained Convolutional Neural Network (CNN) from Google-Teachable-Machine (GTM), relying on MobileNet, was leveraged in training 4050 different frames over 27 classes. The Google-Text-To-Speech (gTTS) library parsed the predicted letters and converted them into a speech file. The Batch Normalization layers and Dropout layers in the CNN yielded a 77 percent average testing accuracy across 25 epochs. These results indicate an opportunity to reliably equip hearing impaired individuals with a representative voice for facilitated communication.