Signvoice – Sign and Speech Communication Web Application

This study has proposed a real-time web-based communication system that aims to bridge the communication gap between deaf and dumb people and the hearing population. It has two main modules. The first module is the speech to text module, which translates the spoken language into the text and generates the images for each letter of the spoken words. The second module is the sign language recognition module, which captures the hand signs from a live video feed and translates the same into text. This system uses WebRTC for live video communication and WebSockets for real-time data transfer, so that the communication can be in real-time. The backend is built with FastAPI, so that it can process the audio input and relay the text and the images to the frontend. CNN is used to identify and translate sign language motions. The frontend is built with React.js, so that it can provide real-time voice and sign processing and an easy user interface. This system is built in such a way that it is cheap, can run on any platform, and is completely independent from the database and other paid services, and it can operate only in English. The system has been tested and it has been found that it has higher accuracy, lower latency, and real-time adaptation, so it can improve the accessibility of communication for everyone.