A Novel Approach for Lightweight Sign Language Recognition Leveraging Google Mediapipe and Deep Neural Net

The advent of technologies like Deep Learning has revolutionized human interaction, transcending language and disability barriers. Sign Language Recognition (SLR) systems have emerged as vital tools, facilitating seamless communication between individuals and between humans and machines through Interfaces. However, existing SLR systems pose significant challenges, including the demand for high accuracy, low latency, simplicity in development and deployment, lightweight design for edge compatibility, and efficient resource utilization during training. Addressing these challenges necessitates developing a system that balances training data accuracy, ease of training, and resource efficiency. Conventional Deep learning approaches often rely on high-dimensional image datasets for training HGR models, resulting in trade-offs between latency, accuracy, complexity, and resource requirements. To mitigate these challenges, this paper harnesses the capabilities of Google Mediapipe to extract a 21-point hand landmark vector from each training image. By leveraging this low-dimensional data, we train model that are accurate, lightweight, less complex, and exhibit low latency, making them compatible with edge devices. The hand gesture images were collected for several gestures, such as peace, okay, and stop. These images will be converted to landmark vectors using Mediapipe. These landmark points were used to train the Deep Neural Network architecture developed in Tensorflow. The model was trained with each hand gesture assigned with a common word in English and the proposed model was tested with a live webcam stream. The proposed model is developed in Python in the Google Colab IDE with the help of libraries and frameworks like OpenCV, Tensorflow and Google Mediapipe. The proposed model outperforms current state-of-the-art approaches by achieving an accuracy of $\mathbf{9 4. 8 8 \%}$.