Machine learning approaches for efficient recognition of Brazilian Sign Language

Context: The Brazilian Sign Language (LIBRAS) is the primary language for the Brazilian Deaf community. However, the lack of LIBRAS interpreters presents challenges for the inclusion of individuals with hearing impairments in society. Additionally, learning a sign language can be as difficult for hearing individuals as learning a foreign language. Problem: There is no official translator between Brazilian Portuguese and LIBRAS, making communication between hearing and Deaf individuals challenging. Solution: This study presents machine learning algorithms that can recognize 44 LIBRAS signs and translate them in real time to Brazilian Portuguese. IS Theory: This work is grounded in the Diffusion of Innovations Theory, focusing on how new technologies are adopted within communities. The proposed LIBRAS recognition system serves as an innovative solution to enhance communication between hearing and Deaf individuals. Method: This research adopts an experimental approach with quantitative and descriptive analyses. Data were collected from three different datasets, representing a total of 44 distinct LIBRAS signs. MediaPipe Hands was used to extract the spatial coordinates of hand movements. Then, some machine learning models were employed to recognize the signs, comparing their performance based on accuracy, F1 Score, precision, and recall metrics. Summary of Results: The results indicated that the ExtraTree algorithm was the best choice for the 44 signs dataset, achieving an accuracy of 98.01%, with an F1 Score of 98.01%, precision of 98.02%, and recall of 98.01%. Contributions and Impacts in IS Area: This paper contributes a low-cost approach for sign language recognition that requires only a camera to detect hands and recognize signs.