Tumor detection in breast cancer pathology patches using a Multi-scale Multi-head Self-attention Ensemble Network on Whole Slide Images

Breast cancer (BC) is the most common type of cancer among women globally and is one of the leading causes of cancer-related deaths among women. In the diagnosis of BC, histopathological assessment is the gold standard, where automated tumor detection technologies play a pivotal role. Utilizing Convolutional Neural Networks (CNNs) for automated analysis of image patches from Whole Slide Images (WSIs) enhances detection accuracy and alleviates the workload of pathologists. However, CNNs often face limitations in handling pathological patches due to a lack of sufficient contextual information and limited feature generation capabilities. To address this, we propose a novel Multi-scale Multi-head Self-attention Ensemble Network (MMSEN), which integrates a multi-scale feature generation module, a convolutional self-attention module, and an adaptive feature integration with an output module, effectively optimizing the performance of classical CNNs. The design of MMSEN optimizes the capture of key information and the comprehensive integration of features in WSIs pathological patches, significantly enhancing the precision of tumor detection. Validation results from a five-fold cross-validation experiment on the PatchCamelyon (PCam) dataset demonstrate that MMSEN achieves a ROC-AUC of 99.01% ± 0.02%, an F1-score of 98.00% ± 0.08%, a Balanced Accuracy (B-Acc) of 98.00% ± 0.08%, and a Matthews Correlation Coefficient (MCC) of 96.00% ± 0.16% ( p < 0 . 05 ). These results demonstrate the effectiveness and potential of MMSEN in detecting tumors from pathological patches in WSIs for BC. • MMSEN Ensemble Learning: Combines CNNs and innovative self-attention for tumor detection. • 1D Convolution in CSAM: 1D convolutions in self-attention enhance pixel-level relations. • Enhanced Feature Integration: Merges multi-scale and self-attention for detection. • Training Strategy: Uses transfer learning and tailored rates to boost CNNs.