Hybrid VGG16 and Vision Transformer Approach for Breast Cancer Classification in Ultrasound Imaging with Explainable AI

Breast cancer is still a major worldwide health issue, and better patient outcomes depend on early identification. To assist with the classification of breast ultrasound images, the research proposes a hybrid deep learning technique that integrates the benefits provided by VGG16 and Vision Transformers. Our strategy utilizes a Vision Transformer's Multi-Head Attention mechanism, which allows models to capture global contextual relationships better than traditional convolutional models. With respect to medical image analysis, the method also covers significant aspects of model explainability, class imbalance, and small sample size problems. The model is proposed and tested on the BUSI dataset, and the results show that the model is the best because it performed with an accuracy of 98% for all the categories including normal, benign, and malignant. With F1 scores of 0.98 for benign and 0.97 for malignant and normal cases, the model balance is exemplary. The use of dropout regularization combined with the Adam optimizer and VGG16 weights further increases generalization and robustness. This combination is particularly beneficial for clinical applications because it enhances the interpretability of the model while achieving high accuracy.