Indian Sign Language Human Activity Recognition

Abstract—Indian Sign Language (ISL) is an important form of communication in India for the hearing and speech-impaired community. However, due to a lack of understanding by the general population, considerable communication barriers exist. This project attempts to develop an image-based ISL recognition system using Convolutional Neural Networks (CNNs) for automatic detection and classification of hand gestures. The ISL dataset is obtained from Kaggle containing gesture images that are labeled. The data are preprocessed and augmented with TensorFlow’s ImageDataGenerator to improve the generalization of the models. The classification system is developed as a - Sequential CNN model - implemented using several convolutional and pooling layers followed by dense layers - and is trained and validated on the dataset with a split of 80-20. Final results demonstrate the models generalizability with high accuracy identifying ISL gestures and producing an acceptable accuracy on unseen images and datasets. The findings illustrate the capabilities of applications of deep learning and computer vision in assistive technologies for reducing barriers in communication. The proposed system will serve as a basis for capturing ISL in real-time to be integrated into ISL to a text translation tool and potential other communication assistive technologies.