Prediction of pathological complete response to neoadjuvant therapy in breast cancer using deep learning with multi-modal radiological image and biopsy whole slide images: a two-center study

To develop a predictive model utilizing deep learning by mammography (MG), ultrasound (US), magnetic resonance imaging (MRI), and biopsy whole-slide images (WSI) to evaluate pathological complete response (pCR) of primary lesion after neoadjuvant therapy (NAT) in breast cancer patients. The retrospective investigation was conducted on 397 patients with pathologically proven invasive breast cancer from two Institution. 300 patients from Institution 1 (92 pCR, 208 non-pCR) were randomly divided into training and validation cohorts in a 3:1 ratio. From Institution 2, 97 patients (34 pCR, 63 non-pCR) were included as the independent external test cohort. All patients underwent mammography, ultrasound, and MRI within one month before NAT. ResNet50, DenseNet169, and DenseNet121 convolutional neural networks were employed to develop MG, US, and MRI deep learning model (DLM), respectively. Biopsy slides were collected for evaluation, and the pathology DLM was developed using ResNet34. Clinicopathological data and radiographic characteristics were analyzed by univariate and multivariate logistic regression analysis to determine independent predictors of pCR after NAT, allowing to the establishment of the clinical model. The comprehensive model was assembled by combining MG, US, MRI, pathology, and clinical models. The evaluation of models was conducted by receiver operating characteristic curves, area under the curve (AUC), confusion matrices, accuracy, sensitivity, specificity, and F1 score. Decision curve analysis (DCA) was employed to evaluate the clinical benefits, and the DeLong test was utilized for comparing AUC values between models. Independent predictors of pCR following NAT comprised HER-2 expression status and the minimal ADC value, which were utilized to develop the clinical model. In the training, validation, and external test cohorts, the comprehensive model demonstrated superior performance, with AUCs of 0.870, 0.842, and 0.801, respectively. Compared with the comprehensive model, the DeLong test indicated that the AUC of the clinical model and the mammography model were statistically significant differences (p < 0.05). DCA revealed that the comprehensive model has the highest clinical benefit. The comprehensive model incorporating mammography, ultrasound, MRI, WSI and clinicopathological-radiological features showed certain predictive value for predicting pCR of primary lesion in breast cancer patients following NAT, and the predictive value of the comprehensive model was better than single modality model.