Baseline T2-based all-in-one automated deep learning management system for neoadjuvant therapy efficacy and prognosis in locally advanced rectal cancer

Background: Current methods for assessing the efficacy of neoadjuvant therapy and predicting patient survival and recurrence risk in locally advanced rectal cancer prior to treatment are limited. This study aimed to develop a multi-module automated deep learning system to evaluate the pathological complete response (pCR) and prognosis of neoadjuvant therapy in patients at baseline. Methods: This multicenter study retrospectively included T2-weighted images from a total of 354 patients with pathologically confirmed locally advanced rectal cancer who received neoadjuvant therapy from 2018 to 2022. The long-term prognosis of patients was also recorded, including overall survival (OS) and disease-free survival (DFS). Center I contained 227 patients as the development cohort, and centers II and III contained 72 and 55 patients as the external test cohorts, respectively. Lesion delineation was performed manually by a radiologist with ten years of experience. Image preprocessing, including N4 bias field correction, resampling, and image normalization, was performed prior to analysis. The study consisted of four main modules; first, an advanced 3D-SwinUNETR segmentation module was constructed and trained using a development cohort. After 15000 iterations, the best model is saved and the corresponding prediction mask is generated. Second, based on the generated prediction masks, three different analysis modules are used. First, a 3D-ResNet-152 model is constructed and trained with the development cohort to predict pCR for patients. Second, based on the 3D-ResNet-152 model framework, quantitative deep features (QDLs) were extracted, and a prediction model was constructed to evaluate pCR through a feature screening method. Third, radiomics features (RFs) are extracted, and a predictive model is constructed to evaluate pCR through feature screening methods. Finally, a fusion model was constructed based on the three modules to assess neoadjuvant therapy efficacy, OS, and DFS. Dice similarity coefficients (DSC) was used to evaluate the segmentation model, Area under the receiver operating characteristic curve (AUC) was used to assess the predictive performance of neoadjuvant efficacy, Kaplan Meier was used for DFS and OS analysis, and Log-rank was used to test for statistical differences. Findings: In the segmentation module, the DSC for the two external cohorts was 0.703±0.020 and 0.698±0.025, respectively. The fusion model demonstrated the best efficacy for assessing pCR, achieving AUCs of 0.756 and 0.751. Log-rank analysis indicated the fusion model's effectiveness in risk-stratifying OS, with p-values of 0.033 and 0.023, and suggested potential stratification for DFS, with p-values of 0.068 and 0.044. Interpretation: This deep learning-based approach can effectively assess the neoadjuvant therapy efficacy and long-term prognosis at baseline.