Real-Time Sign Language Recognition using Deep Convolutional Neural Networks and MediaPipe Hand Detection

This paper presents a comprehensive real-time American Sign Language (ASL) recognition system that combines deep convolutional neural networks with MediaPipe hand detection for robust gesture classification. A ResNet18-based architecture pretrained on ImageNet was employed and fine-tuned on 87,000 ASL alphabet images, achieving 99.60% test accuracy across 26 letter classes (A-Z). The system addresses critical real-world deployment challenges including background clutter and lighting variations through MediaPipe-based hand detection and isolation. The implementation demonstrates real-time inference at 25-30 FPS on consumer hardware (NVIDIA GeForce MX450 with 2GB VRAM), making it practical for accessibility applications. Key contributions include:- End-to-end pipeline achieving 99.60% accuracy on 26 ASL letter classes- MediaPipe integration addressing background clutter and improving real-world robustness- Comprehensive analysis of laboratory-to-deployment performance gap- Open-source implementation with five documented Jupyter notebooks- Real-time performance (25-30 FPS) on consumer-grade GPU hardware The complete pipeline—from data preprocessing through model training to real-time inference—is documented through five Jupyter notebooks, providing a reproducible framework for sign language recognition research. All code, trained models, and configuration files are publicly available on GitHub.