Predicting Recurrence in Locally Advanced Rectal Cancer Using Multitask Deep Learning and Multimodal MRI

Purpose To develop and validate a deep multitask network, MultiRecNet, for fully automatic prediction of disease-free survival (DFS) in patients with neoadjuvant chemoradiotherapy (nCRT)-treated locally advanced rectal cancer (LARC). Materials and Methods This retrospective study collected clinical information and baseline multimodal MRI (T2, apparent diffusion coefficient [ADC], <i>D</i>_app, and <i>K</i>_app) data from patients with LARC after nCRT at three centers between October 2011 and May 2019. Patients from centers 1 and 2 were divided into training, validation, and internal testing sets, while patients from center 3 served as the external testing set. MultiRecNet is capable of simultaneously performing segmentation, classification, and survival prediction tasks within a single framework. Multiple combinations of data from different clinical stages (pretreatment and postoperative) were input into MultiRecNet to generate different models and identify the model with optimal performance. Evaluation metrics included the Dice similarity coefficient (DSC), the area under the receiver operating characteristic curve (AUC), and the Harrell concordance index (C-index) for the segmentation, classification, and survival prediction tasks, respectively. Results The study included 445 patients: 261 in the training set (median age, 60 years [IQR, 53-67 years]; 172 male), 37 in the validation set (median age, 61 years [IQR, 55-68 years]; 30 male), 75 in the internal testing set (median age, 60 years [IQR, 51-67 years]; 45 male), and 72 in the external testing set (median age, 55 years [IQR, 49-61 years]; 38 male). In the internal testing set, the best model based on MultiRecNet (the All model, with T2-weighted imaging, ADC, <i>D</i>_app, <i>K</i>_app, pretreatment clinical indicators, and postoperative pathologic indicators) achieved a DSC of 0.72 for tumor segmentation, an AUC of 0.97 (95% CI: 0.92, >.99) for recurrence or metastasis classification at 3 years, and a C-index of 0.92 for DFS prediction. In the external testing set, the model continued to perform well for survival prediction (C-index = 0.81, <i>P</i> < .001). Conclusion The MultiRecNet-based model enabled prognostic prediction in a fully automated end-to-end manner in patients with LARC following nCRT. <b>Keywords:</b> MR-Imaging, Abdomen/GI, Rectum, Oncology <i>Supplemental material is available for this article.</i> Published under a CC BY 4.0 license.