Vision-based Classification of Sign Language Words using Deep Learning

It might be difficult for people who do not understand sign language to communicate with deaf or mute people. This often results into Misunderstandings and feelings of exclusion. This paper presents a system that can identify and categorize hand gesture photographs to determine the words they convey to solve this problem. Five transfer learning models were trained and tested on a custom-made dataset consisting of 112 classes. Training accuracy, testing accuracy, precision, recall, and F1-score were the five primary performance metrics used to assess the various deep learning models. EfficientNetB3 produced the greatest results, with a 91.96% validation accuracy. ResNet50, on the other hand, had the lowest validation accuracy, at 46.21%. These findings demonstrate that the creation of accessible communication systems can be successfully aided by deep learning approaches. Using image-based gesture detection, our approach facilitates improved inclusion and engagement between the hearing and deaf or mute communities.