Bridging The Gap: ISL Fingerspelling to Text, Sentiment Analysis and Language Conversion

Sign language plays a important role in facilitating communication for people with disabilities. However, even in modern times, there remains a noticeable gap in communication among people who are fluent in sign language and those who are not. Moreover, traditional day to day sign language systems more often rely solely on direct word recognition but this method falls short for when individuals seek to convey proper nouns, such as names. To solve this issue, this research introduces an innovative initiative for real-time Indian Sign Language (ISL) fingerspelling recognition to predict letters and combining them to form words, with a spacebar on a keyboard to indicate spaces, enabling the capture of complete sentences, Furthermore the system includes text translation and audio synthesis of the translated text in the language preferred by the user while supporting over 30+ languages, thereby addressing communication barriers for the deaf, blind, and non-sign language users. This paper presents a resnet50 model with accuracy of 99.06 and validation accuracy of 99.81 to recognize Indian sign language and a LSTM model for sentiment analysis with accuracy of 97.64% and test accuracy 97.81 %.