A Multimodal Translation Interface for Sign Language

This project report describes the development of a “BSL Video Carousel for Sign language” application—a multimodal system designed to facilitate the translation of gestures, spoken language, and text into British Sign Language (BSL) videos. The system leverages computer vision for real-time gesture recognition using a webcam, speech-to-text transcription for voice inputs, and natural language processing (NLP) techniques to simplify and convert textual inputs into a list of essential BSL keywords. These keywords are then used to retrieve corresponding sign videos from an online repository. This report outlines both the existing approaches and the proposed solution, provides an in-depth discussion of the implementation architecture, and evaluates the system’s performance and limitations.