Sign Language Detection Using Faster R-CNN with Resnet50 Backbone

Sign language is the primary means of communication for deaf individuals, yet not everyone is proficient in its use, creating communication barriers. With the advancement of artificial intelligence, there is potential to utilize AI for detecting and interpreting sign language, thus facilitating better communication. This research aims to compare the performance of various object detection models to identify the most effective one for detecting sign language. Our study focuses on evaluating four different models: Faster R-CNN with ResNet50 backbone, Faster R-CNN with MobileNetV3 backbone, SSDLite, and YOLOv8. Using a dataset of 1,728 images representing American Sign Language with multiple classes, we preprocess the data and assess the models based on their Mean Average Precision (mAP) values. Our goal is to determine which model provides the highest accuracy and efficiency in detecting sign language. The evaluation revealed that the Faster R-CNN with ResNet50 backbone achieved the highest mAP value of 0.87 and 0.66, followed by Faster R-CNN with MobileNetV3 backbone at 0.88 and 0.63, YOLOv8 at 0.79 and 0.66, and SSDLite at 0.73 and 0.53. This research demonstrates the potential for AI to bridge communication gaps and improve sign language detection methods. Future studies could enhance these models by increasing the dataset size and incorporating more diverse sign language variations to improve performance and reliability further.