Brain Tumor Diagnosis Using Vision Transformer and Optimized ViT with Hybrid Optimizers, and XAI Integration

Brain tumors critically affect human life and mostly fall into glioma, meningioma, and pituitary tumor categories. Detection in early stages and precise categorization lead efficiently to successful diagnosis and lowered healthcare expenditure. A classification system based on deep learning applying the ViT architecture to discriminate brain MRI images among these three types of tumors is introduced in this work. The initial brain tumor dataset in the .mat file format was resized to the .jpg file type and from 512–512 pixels to 224–224 pixels for suitable compatibility for processing using deep learning models. The dataset consisted of 3064 contrast-enhanced T1-weighted MRI images, and the model had a classification accuracy of 93.8%. To improve the models performance and in this research, we used the RAdam and LookAhead optimizers and assessed the model’s performance using the 5-Fold cross-validation technique. In this study, the proposed model’s accuracy was obtained at 96.75%, 98.73%, 98.01%, 97.47%, and 98.37% through the use of the ViT, thus deriving a cross-validation accuracy of 97.87%, the suggested framework is supported by evidence that confirms its robustness and ability to generalize. To facilitate benchmarking, ViT was compared with various CNN models, such as VGG16 with Random Forest (VGG16-RF), VGG19, EfficientNet-B3, and MobileNet-Large, and these reached accuracies of 89% to 98%. Also, in order to provide more interpretability and trust from clinicians, XAI models like Grad-CAM, Grad-CAM++, LIME, and SHAP were incorporated and provided visual and feature level explanations of the model predictions. The results confirm that the performance of ViT beats conventional CNN models for the brain tumor classification and provides for transparent and interpretable decision making, and its excellent potential for real-world clinical workflows incorporation is emphasized.