Advancing real-time sign language detection for deaf and hearing-impaired communities: a customized YOLOv8 approach with tailored annotations in computer vision

Sign language is a crucial mode of communication for deaf and hearing-impaired individuals, especially in the context of digital communication platforms such as Zoom and Google Meet. While computer vision technologies have seen significant progress, achieving robust, real-time sign language detection remains a key challenge for enhancing communication accessibility. This study aims to address this gap by exploring the research question: Can a customized object detection system combining YOLOv8 and InceptionV3 provide accurate and real-time sign language recognition? To develop a real-time sign language detection system, we employed the YOLOv8 object detection architecture enhanced with features extracted from an InceptionV3 convolutional neural network (CNN). A specialized dataset was constructed with tailored annotations to support fine-grained sign detection and classification. The combined model was trained and evaluated using this custom dataset, with particular focus on detection accuracy and inference speed. Performance metrics included mean Average Precision (mAP), F1-score, and inference time. The proposed YOLOv8-InceptionV3 model demonstrated high accuracy and real-time capabilities. It achieved a mean Average Precision at 50% IoU (mAP50) of 99.5%, a mAP50-95 of 86.2%, and an F1-score of 95.40%. The average inference time per image was recorded at 4.6 milliseconds. These results surpass those of prior models developed with smaller or less specialized datasets. The integration of YOLOv8 with InceptionV3 provides a powerful solution for real-time sign language detection. The system’s high accuracy and low latency make it well-suited for deployment in digital communication tools, enhancing accessibility for deaf and hearing-impaired users. This approach presents a significant step toward more inclusive and intelligent communication platforms.