Multi-modal feature fusion and classification of histopathological images using ensemble learning

Early and correct detection of cancer regions in histopathology images is important for timely diagnosis and treatment. In this work, we proposed a Explainable Artificial Intelligence based convolutional Neural network (CNN) framework for the classification of lung and colon cancer. The framework combines deep features from a CNN with hand-crafted features in a single hybrid feature pipeline. From each image, we first extract a 512-dimensional feature vector using a CNN. We then join this vector with 6084 HOG features and 10 LBP features to obtain a 6606-dimensional feature vector. To remove repeated and less useful information, we apply IPCA before training the models. On these reduced features, we train several classifiers: Random Forest, Support Vector Machine (SVM), Logistic Regression, and a custom MLP. Their predictions are then combined using a stacking-based ensemble so that the final model can use the strengths of each individual classifier. To make the system easier to understand for clinicians, we use SHAP-based explanation to show which features are important and how they affect the model output. The performance of the proposed ensemble framework is evaluated on LC25000 histopathology dataset and it obtains an accuracy of 99.38%, which is higher than several recent baseline methods. Furthermore, a detailed ablation study is performed to explain the role of each features extractor and combinations in precise classification. These finding concludes that combining deep and hand-crafted features with ensemble learning and simple explanation tools can lead to a reliable and accurate cancer classification system.