Benchmarking CNN and Cutting-Edge Transformer Models for Brain Tumor Classification Through Transfer Learning

Brain tumor is a serious disease that can lead to fatal consequences. Moreover, there are different types of brain tumor with different progression rate and severeness. Thus, brain tumor classification is an essential task in medical diagnosis. Convolutional Neural Network (CNN) based deep learning methods shown great potential brain tumor classification using Magnetic Resonance Imaging (MRI) scans of brain. Recent development in transformer based deep learning models have shown promising outcome in image classification task. However, these models have not yet much explored in the application of brain tumor classification. In this paper, we employ seven advanced models which utilizes transformers or mimics the self-attention operation of transformers through their design, for brain tumor classification. To assess their performance for this particular task, five conventional CNN based methods have also been applied and compared with these models. To make our assessment more coherent and comprehensive we perform the comparison for four different datasets. The results indicate that transformer-based advanced models do not provide a distinct advantage over conventional CNN-based models. The CNN-based ResNet-50 performs well overall, especially with smaller datasets. For larger datasets, transformer-based models generally perform better than CNN-based models, although the difference is not significant.