Empowering the Deaf Community Through a Vision-Based Approach to Bangla Sign Language Numerals Recognition

Sign language is a vital mode of communication for individuals who are deaf or hard of hearing. Recognizing Bangla Sign Language (BSL), particularly numerals, presents unique challenges in the fields of computer vision and deep learning due to variations in skin tone, hand positions, image quality, and environmental conditions. While sensor-based recognition techniques offer higher accuracy, they are often costly and inaccessible. This study proposes an efficient, cost-effective method for Bangla sign numeral recognition using hand images captured via standard web cameras. The MediaPipe algorithm was employed to extract hand joint positions from RGB images, which were then used to compute two feature types: distances between key points and angles between vectors and 3D axes. Classification was performed using Support Vector Machine (SVM) and Light Gradient Boosting Machine (LightGBM) algorithms. The experiments utilized the BHSGR-Net dataset, consisting of one-handed representations of Bangla sign numerals from 30 participants performing 10 distinct digits. The results revealed that SVM achieved 100% accuracy with angle-based features, while LightGBM obtained 95.60% accuracy using a combination of distance-based and angle-based features. The proposed approach is computationally efficient, requires no additional sensors or equipment, and demonstrates superior performance compared to previous methods, making it a promising solution for automatic Bangla sign numeral recognition.