A deep ensemble-based framework for the prediction of oral cancer through histopathological images

Early diagnosis and treatment for cancer can save lives and is a primary worldwide health concern. This is also true for Oral Cancer (OC), highlighting how crucial early intervention is. OC, specifically Oral Squamous Cell Carcinoma (OSCC), is a complex malignancy with a high mortality rate. One group of diseases that dentists can diagnose and treat is OSCC, which are found in the oral cavity. Deep Learning (DL) algorithms have shown promise in the identification and prediction of OC due to their ability to analyze large datasets and detect subtle patterns in buccal tissue. This work introduces a lightweight ensemble-based DL network for the prediction of OC. The proposed methodology incorporates various DL techniques such as Convolutional Neural Network (CNN), Bidirectional Long Short Term Memory (Bi-LSTM), and Bidirectional Gated Recurrent Unit (Bi-GRU) for extraction of deep features. Here, we have developed the CNN and Bi-LSTM blocks to extract the spatial and contextual features from the histopathological images. Also, the proposed Bi-GRU block increases classification performance by exploiting spatial and sequential features to better capture dependencies within image data when paired with CNNs. After the deep features are extracted from the proposed CNN, Bi-LSTM, and Bi-GRU blocks, they are amalgamated to form a new set of deep features. To forecast the probability of OC among unnoticeable patients during screening, this study uses a soft-voting ensemble classifier to classify after the deep features have been extracted from the proposed deep network based on five baseline classifiers including Logistic Regression (LR), Decision Tree (DT), k -Nearest Neighbors ( k -NN), Support Vector Machine (SVM), and Stochastic Gradient Descent (SGD). For experimental verification here, we have used the two publicly available OC image datasets, which are the Histopathological Oral Cancer Detection Dataset (HOCDD) and the Histopathological Image Repository of Normal Epithelium of Oral Cavity and OSCC Dataset (HRNEOCD). By effectively leveraging deep features, the proposed ensemble-based classification model achieves remarkable accuracy scores of 98.34% for the HOCDD dataset and 97.89%, 98.76% for the HRNEOCD Set-1, Set-2, dataset, showcasing its robust predictive capabilities and its potential for reliable OC prediction. This study highlights the importance of integrating DL algorithms with conventional classifiers to enhance the performance of OC prediction models. • A lightweight ensemble-based deep network for the prediction of oral cancer. • CNN and Bi-LSTM extract spatial and contextual features from histopathological images. • Bi-GRU improves classification by capturing spatial and sequential dependencies. • Deep features from CNN, Bi-LSTM, and Bi-GRU are amalgamated for better accuracy. • Soft-voting classifier gives reliable prediction for unnoticeable & noticeable cases. • It is computationally efficient compared to other transformer-based models.