A Robust Vision-Based Dynamic Sign Language Recognition Using a Hybrid CNN-LSTM Model

To recognize meaningful movements of hand gestures, Sign language recognition is important. It is a significant solution in intelligent communication between the deaf community and hearing societies. The general population’s lack of proficiency in sign language impedes meaningful interactions with individuals who depend on it. Although many studies have been conducted on this topic, the development of static and dynamic sign language recognition models continues to hold significant challenges in the field of research. This paper presents a novel framework for implementing a vision based Sinhala Sign Language recognition model using deep learning approaches. Our proposed methodology includes image acquisition, pre-processing, detection, and classification stages for a complete recognition. In the classification stage, the CNN-LSTM model achieves 94% accuracy for around 20 signs. This method identifies and interprets unique common signs and gives a meaningful sentence in text. The proposed approach shows promising results for interpreting both static and dynamic signs of Sinhala sign language.