Predicting Indian Sign Language Gestures Using Deep Learning and Machine Learning Techniques

Indian Sign Language is the language used by people with disabilities in India. Unfortunately, the general public's ignorance of sign language makes it difficult for the especially abled and the general public to communicate. We are releasing a strategy to reduce this difference. We have adopted two distinct strategies to deal with this problem. first by employing the depth+RGB data acquired with a Microsoft Kinect to make gesture predictions in real-time. To segment the hand region from the RGB-D camera data, we used affine transformation and 3D reconstruction to map the RGB and depth information. Convolutional neural networks were trained with inputs that were segmented hand images or videos. With 50 static Indian Sign Language hand gestures used for training, a 97.18% classification accuracy was achieved on test data. Furthermore, this model achieved a 97.71 % classification accuracy when we transferred learning American Sign Language. A convolutional kernel and an LSP were used to train twenty dynamic gestures. This model's classification accuracy was 98.88%. Nevertheless, as soon as we implemented this system, we found that the methodology was flawed. Essentially, it is absurd to need to carry around both a bulky Microsoft Kinect and a device capable of performing the computation required for human-to-person interaction. We attempted to address this issue by using semantic segmentation of the hands. U-Net and ResNet 101 were combined.