Deep Learning - Based Real Time Sign Language Translator Using YOLO

Abstract: This paper presents a comprehensive Real-Time Sign Language Translator that bridges the communication gap between hearing-impaired individuals and the hearing population. The proposed system integrates two major functionalities: (1) real-time detection of alphabets and numbers using a YOLOv8-based object detection model, and (2) sentence-level gesture recognition along with speech-to-text translation using deep learning and natural language processing techniques. The system utilizes YOLOv8, OpenCV, and MediaPipefor visual detection and classification of hand signs, and *Flask for creating a responsive web-based interface. In the first module, the system supports the recognition of 36 static signs, including A–Z alphabets and digits 0–9. In the second module, 32 sentence gestures are recognized in real time and translated into meaningful sentences. Additionally, the system captures voice input using a microphone, converts it to text via speech recognition, and translates it into Tamil or Hindi. The platform includes functionality for live detection, screenshot capture, video recording, and multilingual output display, offering an inclusive and practical solution for accessible communication