Real-time sign language translation using CNN and mediapipe with multilingual speech output

The communication gap existing between the hearing-impaired and the community in the larger society poses serious socio-economic challenges. This paper suggests the Real-Time Sign Language Translator that can fill this gap and gives the ability to transform hand gestures into the spoken language within Real-Time, with the help of Convolutional Neural Networks (CNN) and Media Pipe. This system streams the live video with a webcam, identifies the landmarks of the hand with Media Pipe, and classifies the hand in terms of the static gestures to corresponding alphabets or words with the already trained CNN model. It then transcribes the identified text into multilingual speech output which is a good way of communicating in English and various Indian languages. With an accuracy rate of more than 98.50%, the system is low latency and gives a user-friendly interface. This translator is scalable, affordable and easy to access, which is why it is an incredible assistive technology to the hearing-impaired community. Suggestions of future improvements can entail dynamic gesture support, sustained sentence inference, and mobile platforms implementation.