Real-Time Speech-to-Text Holographic Communication for the Deaf Children and Elderly

In the field of assistive technologies, innovative solutions that cater to the needs of individual with disabilities are paramount for fostering inclusive communication environments. The proposed work introduces an innovative real-time speech-to-text conversion system using holographic display technology to aid communication for the deaf and hard-of-hearing (DHH). The system incorporates two complementary speech recognition approaches as each contribute uniquely to improve the system's overall performance and accuracy in different ways and also reduce the latency. First approach is a conventional speech recognition module and another is an enhanced version of the Wav2vec2 model, which has been retrained to improve its efficacy with diverse speech patterns. Hosted on a Raspberry Pi and interfaced with an USB microphone, the system captures spoken language effectively. Upon recognition, the text is instantly displayed on a holographic screen made up of LED matrices and set against an acrylic glass surface, creating a visually engaging representation of floating text. The integration of these two recognition methods enhances the system's accuracy and reliability in processing real-time speech. By providing a visual output of spoken words, this system not only bridges communication gaps but also enriches the interaction experience within various social and professional settings. The application of holographic technology in this context is pioneering, offering substantial promise for developing accessible communication tools that are both functional and inclusive. This work underscores the potential of advanced display and speech processing technologies to transform interactions for individuals with hearing impairments, fostering greater inclusivity and engagement in everyday communications.