Performance Comparison of Different Versions of YOLO for Indian Sign Language Captioning in Real Time of Multiple Signers

Communication is an essential aspect of life, enabling people to exchange messages. While most individuals can communicate verbally, people who are deaf use different techniques, such as sign language, for communication. To facilitate interaction for the deaf community, Indian Sign Language (ISL) recognition using You Only Look Once (YOLO) architecture has become a prevalent and effective approach for detecting signs in real time using deep learning techniques. For this research, experimentations were performed using various YOLO versions, including YOLOv5, YOLOv6, YOLOv7, YOLOv8, YOLOv9, YOLO NAS, YOLOv10, and YOLO 11. The self-created dataset consists of 115 ISL signs (classes). We used 10 different signers and created 20 to 40 images for each class. The models are tested on 115 ISL signs to recognize the signs accurately in real time. All YOLO versions are trained on our self-created annotated dataset. The results showed that YOLO NAS outperformed the other versions. After fine-tuning some key parameters, YOLOv5, YOLOv8, YOLOv9, YOLOv10, YOLO 11, and YOLO NAS, a mean average precision (mAP) of 0.969, 0.975, 0.974, 0.960, 0.978 and 0.982 respectively with value of 0.50 as intersection over union (IoU) threshold were achieved. These results demonstrate the potential of these models for real-time ISL recognition. Our model also recognises signs of multiple persons in a single frame, which can be further used for ISL conversation in real time.