Real-Time Chinese Sign Language Recognition Based on Convolutional Neural Network

The development of assistive technologies plays a vital role in promoting diversity and accessibility in communication. This study presents an advanced computer vision system designed to accurately detect numerical gestures in Chinese Sign Language (CSL). We propose a novel hybrid framework that integrates MediaPipe for real-time hand detection and keypoint extraction with a YOLOv5-based CNN classifier for robust digit recognition. A custom dataset comprising 1410 images of CSL digits (0–9), collected under varying hand poses, lighting conditions, and backgrounds, was constructed to enhance model generalization. Preprocessing included grayscale conversion, Haar-cascade-based hand region extraction, and normalization. Experimental results demonstrate strong performance, achieving 99.1% accuracy, 98.7% precision, 98.9% recall, 98.8% F1-score, and [email protected] of 0.99 on the test set. The system further exhibits real-time capability, maintaining low-latency inference during live video streams. These findings confirm the effectiveness of combining MediaPipe and YOLOv5 for CSL digit recognition and underscore the potential of deep learning-based assistive technologies to improve communication accessibility for the deaf and hard-of-hearing community.