A deep learning framework for Ethiopian sign language recognition using skeleton-based representation

This study proposes an environment- and signer-invariant sign language recognition model. The model first extracts skeletal key-points from the signer via MediaPipe, which is Google's cross-platform pipeline framework that helps to detect and track human poses, facial landmarks, and hands. After preprocessing the skeletal key-point information, feature extraction and learning are performed via deep learning architectures: a convolutional neural network followed by long short-term memory (CNN-LSTM), long short-term memory (LSTM), bidirectional long short-term memory (BiLSTM), and gated recurrent units (GRUs). This study proposes a deep learning framework for recognizing Ethiopian sign language (EthSL) via skeleton-based features extracted via the MediaPipe Holistic. A dataset of 5600 annotated sign videos was constructed and used to evaluate four deep learning models, namely, CNN-LSTM, LSTM, BiLSTM, and the GRU, which achieved 94% accuracy in signer-dependent settings and 73% accuracy in signer-independent settings. The results demonstrate the model's potential for scalable, low-cost, and real-time EthSL recognition in unconstrained environments. This study attempted to increase the independence of ASLR models to some level. However, further studies are needed to identify continuous signs in a fully open environment. Therefore, the technique implemented to detect and track key points in this study should be further investigated to recognize continuous EthSL.