Robust and Multicenter Detection of Breast Malignancies Using Vision Transformers in Digital Breast Tomosynthesis Imaging

ABSTRACT Breast cancer detection remains a diagnostic challenge, particularly in dense breast tissues. This study aimed to develop and evaluate an integrative deep learning framework that leverages 3D digital breast tomosynthesis (DBT) to improve accuracy, interpretability, and generalizability of breast cancer classification across clinical settings. This framework focuses on image‐level classification of normal, benign, and malignant cases, rather than lesion‐level detection, utilizing DBT images to enhance accuracy and generalizability. A total of 2589 DBT images were collected from six medical centers, categorized into normal, benign, and malignant cases. The framework incorporated both convolutional neural network (CNN)‐based transfer learning models (ResNet50, InceptionV3, EfficientNet‐B0) and transformer‐based models (Vision Transformer, Swin Transformer). Images underwent preprocessing including normalization, filtering, resizing, and augmentation. All models were trained under a 10‐fold cross‐validation scheme with an external validation set retained from an independent institution. Performance was evaluated using accuracy, recall, precision, F1‐score, and AUC‐ROC. To enhance interpretability, Grad‐CAM and t‐SNE visualizations were generated. Statistical tests (paired t ‐test and Wilcoxon) assessed performance significance, and a radiologist independently scored Grad‐CAM heatmaps to evaluate clinical plausibility. Transformer models consistently outperformed CNN architectures across all metrics. Swin Transformer achieved the highest accuracy (94%) and AUC (96%) in both internal and external validations, particularly excelling in malignant lesion detection. Grad‐CAM and t‐SNE demonstrated superior spatial focus and feature separability in transformer models. Statistical tests confirmed the significance of performance differences ( p < 0.05), and radiologist scoring revealed strong clinical alignment in transformer heatmaps (mean score: 1.8). Computational efficiency analysis indicated faster convergence in transformer models despite higher training costs. The proposed framework offers a clinically interpretable and diagnostically accurate solution for DBT‐based breast cancer detection. Transformer architectures, combined with advanced visual analytics, demonstrate potential for real‐world deployment and clinical decision support.