Real-Time Marathi Sign Language Recognition using Deep Learning Techniques

Communication is the most important thing for deaf and dumb people, where sign language plays an essential role in aiding them to communicate with others, however to teach us this form of communication was a challenging task. This work attempts to bridge this communication gap by performing a comprehensive study on real-time Marathi sign language detection. In this work, we have developed a custom dataset devoted to Marathi sign language gestures and investigated various deep learning models such as Convolutional Neural Networks (CNN), VGG-16, MobileNetV2, InceptionV3 for accurately identifying the signs. Rigorous experimentations were carried out and VGG-16 as the final model, boasting a validation accuracy is 95.55% & testing accuracy is 97.38%. In addition, we implemented an easy-to-use GUI which captures the most possible hand gestures in real-time and maps them to Marathi letters correctly. This has the potential to greatly enhance deaf and dumb people’s communication exchanges with ordinary society.