Word Level Ethiopian Sign Language Recognition Using Hybrid CNN-LSTM Model

In this research, we propose a hybrid CNN and LSTM model for word-level Ethiopian sign language recognition. The recognition system has four major components: preprocessing, feature extraction, sequence modeling, and training. Image frame extraction was performed on the collected word video dataset, followed by keyframe selection using the k-means clustering algorithm. Image concatenation and image resizing were applied to the selected keyframe images. The final dataset is downsized into 1500 images, which is 150 images for each class. CNN was utilized to extract the important features through convolutional layers, pooling layers, and dropout functions were used to overcome overfitting issues. Following feature extraction, sequence modeling was performed by using LSTM with LSTM layers, a dense layer, and a SoftMax classifier to recognize words in both predicted and actual classes. The system was tested by the dataset that we collected for this purpose. The experiment demonstrates that the proposed hybrid CNN-LSTM model performs well across the training, validation, and testing phases, achieving 98%,97%, and 95% accuracy during model evaluation, respectively. These results demonstrate the recognition success of the model isolated word-level Ethiopian sign language, which plays a step forward in assisting hearing-impaired individuals.