Combining state-of-the-art pre-trained deep learning models: A novel approach for Bangla Sign Language recognition using Max Voting Ensemble

Sign language is a crucial medium of communication for individuals with hearing impairments. Recently, many researchers have been working to develop an automatic sign language recognition system for English, Arabic, and Japanese languages to ease communication complexity between deaf and non-deaf communities. However, few systems have been developed for Bangla sign language (BdSL), and most of the existing ones may face difficulties in achieving satisfactory performance. While recent advances in deep learning have dramatically improved image classification tasks, including sign language recognition, ensemble methods offer a pathway for further enhancing BdSL identification accuracy. This study introduces a cutting-edge approach employing the Max Voting Ensemble technique for robust BdSL recognition. We have incorporated a range of cutting-edge pre-trained deep neural networks including Xception, InceptionV3, DenseNet121, ResNet50, and MobileNetV2. These models have been extensively trained on BdSL datasets, achieving individual accuracies ranging from 92.96% to 98.81%. Our method leverages the synergistic capabilities of these models by combining their complementary features to elevate classification performance further. In our approach, input images undergo preprocessing for model compatibility. The ensemble integrates the pre-trained models with their architectures and weights preserved. For each Bangla sign under examination, every model produces a prediction. These are subsequently aggregated using the Max Voting Ensemble technique to yield the final classification, with the majority-voted class serving as the conclusive prediction through comprehensive testing on a diverse dataset. Our ensemble outperformed individual models, attaining test accuracy of 96.62% and 99.92% using BdSL-38 and BDSL-49 datasets respectively, thus demonstrating superior BdSL recognition performance and reliability. We evaluated the effectiveness of our proposed method on the BdSL-38 and BDSL-49 datasets to ensure its generalizability. Our ensemble method delivers a robust, reliable and effective tool for the classification of BdSL. By utilizing the power of advanced deep neural networks, we aim to assist healthcare professionals in achieving timely and accurate diagnoses, ultimately reducing mortality rates and enhancing patient outcomes.