Optimizing Sign Language Classification: A Deep Learning Approach with Custom CNNs and Transfer Learning

Hand gestures, facial expressions, and body movements are used to communicate in American Sign Language (ASL), but communication hurdles frequently prevent successful interactions between deaf and hearing cultures. To close this gap, this study investigates deep learning models’ ability to recognize ASL gestures reliably, enabling seamless communication and real-time translation. We compare the performance of three deep learning architectures: a pre-trained ResNet50 model and two custom Convolutional Neural Networks (CNNs). A number of techniques are used to prepare data for modeling, including normalization, and one-hot encoding. To uncover significant features that influence classification accuracy, the Analysis of Variance (ANOVA) test is applied during the feature selection process, aiding in model evaluation. ASL recognition problems such as hand form variations and backdrop interference are discussed, as well as the importance of transfer learning with pre-trained models such as ResNet50. The study’s findings provide useful insights into the potential of deep learning for ASL recognition, as well as future research areas to overcome the identified issues. Using the suggested method in sign language translation and gesture-based communication systems will be highly effective.