Multi-teacher knowledge distillation and ensemble algorithm for efficient brain tumor classification in resource-constrained environments with explainable AI

The deployment of complex deep learning models in resource-limited clinical settings remains a critical challenge for brain tumor classification. This study explores knowledge distillation (KD) and ensembling strategies to enhance brain tumor classification using lightweight convolutional neural networks (CNNs), targeting resource-constrained environments. The research investigates both single-teacher and multi-teacher KD frameworks, utilizing VGG16 and Xception as teacher models to impart knowledge to smaller student models, including Tiny ResNet, Tiny Xception, and Tiny DenseNet. Student models, such as Student-1 (Tiny ResNet), feature a parameter reduction of approximately 83x and 117x times compared to VGG16 and Xception. Moreover, ensembling methods, including Average Ensembling and Weighted Ensembling, were employed to further refine predictive performance further. The multi-teacher KD approach achieved notable accuracy, reaching 96.19 % on Dataset-1 and 95.57 % on Dataset-2, while ensembling techniques yielded up to 98.02 % and 98.15 % for Average Ensembling and Weighted Ensembling, respectively. Furthermore, explainable AI (XAI) techniques, such as Grad-CAM and SHAP, were implemented to improve the interpretability of deep learning models and provide clinicians with insights. The results underscore the potential of combining KD and ensembling to develop robust and efficient AI-driven solutions for brain tumor classification, balancing accuracy, interpretability, and computational efficiency. These advancements make it possible to develop brain tumor classification systems that are accurate and efficient enough for deployment in resource-constrained environments.