Predicting tumor recurrence of locally advanced rectal cancer after neoadjuvant chemoradiotherapy based on multi-task deep-learning model

Motivation: The promising application of deep learning (DL) techniques for prognostic prediction in various tumors has been reported, but mostly with single-task modelsGoal(s): Exploring the use of multi-task DL models to automate the whole process of prediction for rectal cancer patients. Approach: We designed a modality-fusion-based multi-task DL model to concurrently predict tumor volumes, patient relapse state, and patient risk scores based on a combination of multimodal MR images and clinical tabular data. Results: The multi-task DL model achieved favorable predictive performance at the stage of initial diagnosis with automatic lesion identification, and further improved with the inclusion of postoperative pathology indicators. Impact: Multi-tasking DL may be a new approach and orientation to fully automate the process of clinical prediction, and its feasibility is expected to be further explored in other oncology studies in the future.