A Real-Time Sign Language Learning System Using LSTM and Mediapipe

Sign language is a vital communication method for individuals with hearing and speech impairments, yet real-time recognition remains challenging due to gesture variability, occlusions, and motion tracking issues. This study presents a deep learning-based sign language recognition system integrating Mediapipe’s Holistic model for landmark extraction and an LSTM network for gesture classification. The system accurately converts hand movements into text in real time, overcoming the limitations of CNN-based approaches. Advanced preprocessing techniques, including landmark normalization and data augmentation, enhance robustness against lighting variations and occlusions. With a dataset of 50 unique gestures recorded in 100 sequences, the model achieves 92.4% accuracy while maintaining a real-time inference speed of ~50ms per frame. Comparative analysis highlights its superior performance in classification precision and processing speed, making it suitable for assistive applications in education, healthcare, and accessibility.