Benchmarking Deep Learning Models for Breast Cancer Detection: A Comparison of Vision Transformers and CNNs

Breast cancer is a major global health issue, and accurate early detection is critical for improving patient outcomes. Deep learning-based image classification techniques have shown remarkable success in medical imaging, particularly convolutional neural networks (CNNs) and transformer-based models. This study evaluates and compares the performance of Vision Transformers (ViTs) with well-established CNN architectures, including AlexNet, ResNet-50, and VGG-19, for breast cancer image classification. The research aims to investigate whether ViTs can outperform conventional deep learning models in this domain and to analyze their strengths and limitations. The study utilizes a publicly available breast cancer dataset comprising 9,248 images categorized into benign, malignant, and normal classes. The dataset is preprocessed by resizing all images to 224×224 pixels, normalizing pixel intensity values, and applying data augmentation techniques. All models are trained under the same conditions using 80% of the data for training, 10% for validation, and 10% for testing. Performance evaluation is conducted based on accuracy, precision, recall, and F1-score metrics. Experimental results indicate that ResNet-50 achieves the highest classification accuracy (93.62%), outperforming the other models in terms of overall performance. AlexNet, despite having the smallest parameter count, delivers competitive accuracy (88.32%) while being computationally efficient. VGG-19, known for its depth, achieves 87.51% accuracy but has the highest computational cost. ViTs, although promising, achieve a lower accuracy of 87.46%, suggesting that transformer-based architectures may require larger datasets and further optimization to surpass traditional CNNs in medical image classification tasks. This study highlights that CNN-based models, particularly ResNet-50, remain the most effective approach for breast cancer classification in the given dataset. However, ViTs present a potential alternative, and future research should explore hybrid models integrating both CNN and transformer-based architectures to enhance classification performance.