Indian Sign Language Captioning Using Deep Learning Methods

Sign language offers a reliable means of communication for the deaf and mute community. However, its effectiveness diminishes when interacting with individuals who are not familiar with it. Indian Sign Language (ISL), which serves one of the largest user bases, faces similar challenges. This paper proposes a deep learning-based solution for real-time captioning of ISL gestures into coherent English captions. The approach integrates MediaPipe for pose estimation and long-short-term memory (LSTM) networks for sequential data classification. The system efficiently captures gesture dynamics and provides accurate textual captions using pose landmarks extracted from video input. The proposed solution demonstrates the potential to improve accessibility for individuals with hearing or speech impediments.