Real-Time Recognition and Translation of Sinhala Sign Language

Sinhala Sign Language (SSL) is the native visual language of Sri Lanka’s deaf and speech-disabled community, using hand movements, lip movements, facial expressions, and body language, with unique grammar and vocabulary. Existing SSL detection systems aim to identify a limited vocabulary, leaving the development of real-time SSL recognition and translation research largely under explored. This study introduces a real-time framework to accurately detect and translate commonly used words and sentences in SSL into Sinhala text and speech. This framework aims to facilitate effective communication between speech-disabled individuals and non-sign language users. To accomplish this, we create a comprehensive SSL detection dataset, including annotated videos that capture a wide range of Sinhala signs. The proposed real-time framework leverages computer vision techniques to extract precise body landmarks over time frames for accurate sign recognition. A Long short-term memory network is employed to enhance sign classification. Additionally, we developed a user-friendly interface that translates detected signs into Sinhala text and speech. Experiment results demonstrate that the proposed framework achieves an accuracy rate of 96% in recognizing real-time SSL signs, underscoring its potential as an effective tool for day-to-day communication.