Advancing brain tumor classification through pre-trained transformer and transfer learning models

Brain tumor classification by MRI images remains an important yet difficult issue in the field of medical diagnosis. The accuracy of tumor identification significantly influences treatment results; however, the process is hard because of the range of tumor shapes and textures and differences in MRI scans. Since timely and precise diagnosis is paramount for patient care, precise segmentation of brain lesions becomes pivotal as the premise for the next medical decision. Traditional CNNs have registered notable success in medical image analysis. Their natural weakness, though, - a limited receptive field - will generally make it difficult for them to comprehend general spatial relationships in an image, particularly when abnormal areas cover extensive space. In order to overcome this, we investigate Vision Transformers (ViTs), which provide a more comprehensive comprehension of MRI scans by modeling long-range dependencies among picture patches by utilizing self-attention mechanisms. A comparison between a transfer learning strategy utilizing VGG16 and a ViT-based model is presented in this paper. We trained our models on a combined dataset sourced from Figshare, SARTAJ, and Br35H, comprising 7,023 brain MRI images across four classes: glioma, meningioma, no tumor, and pituitary. Experimental results show that the Vision Transformer consistently outperforms the VGG16 model, achieving a test accuracy of 99.6%, with both precision and F1-score reaching 99.59%. Our findings suggest that ViTs hold strong promise for integration into clinical workflows, with potential for generalization across MRI scanners and hospital environments.