Real-Time Translation of Speech to Indian Sign Language to Facilitate Hearing Impairment

Sign language is a visual language utilized by individuals who are deaf as their primary means of communication. Unlike spoken languages, sign language relies on gestures, body movements, and manual communication to effectively convey ideas and thoughts. It can be used by individuals who have difficulty speaking, those who are unable to speak, and by individuals without hearing impairments to communicate with deaf individuals. Access to sign language is crucial for the social, emotional, and linguistic development of deaf individuals. Our project aims to bridge the communication gap between deaf individuals and the general population by leveraging advancements in web applications, machine learning, and natural language processing technologies. The primary objective of this project is to develop an interface capable of converting audio/voice inputs into corresponding sign language for deaf individuals. This is achieved through the simultaneous integration of hand shapes, orientations, and movements of the hands, arms, or body. The interface operates in two phases: first, converting audio to text using speech-to-text APIs (such as Python modules or Google API); and second, representing the text using parse trees and applying the semantics of natural language processing (specifically, NLTK) for the lexical analysis of sign language grammar. This work adheres to the rules of Indian Sign Language (ISL) and follows ISL grammar guidelines.