Sentence-Level Sign Language Recognition Using MediaPipe

According to the World Federation of the Deaf, over 70 million people worldwide use sign language as their primary means of communication. It is estimated that by 2050, more than 700 million people will suffer from hearing loss. Consequently, the demand for a fast and interpreter-free sign language translation medium is increasing daily. Although there are many approaches to word-level sign language translation, there has not been an efficient method to detect sign language on a sentence-level. This research proposes a novel approach to continuously translate sign language using Convolutional Neural Networks (CNN) at the word-level, employing multiple window filters to further process the translation at the sentencelevel. By computing feature maps for each image using Time Distributed layers, the method achieves an overall word-level accuracy of over 99%, a BLEU score of 0.883, and a WER of 0.081 for the sentence-level.