Development and validation of AI delineation of the thoracic RTOG organs at risk with deep learning on multi-institutional datasets

Accurate contouring of thoracic organs at risk (OARs) is essential for minimizing complications in radiation treatment. Manual contouring of thoracic OARs is not only time-consuming but also prone to substantial user variation. To enhance the efficiency and consistency, we developed a unified deep learning (DL) OAR contouring model, DeepOAR, that was trained using multiple partially labeled datasets for segmenting a comprehensive set of thoracic OARs following the Radiation Therapy Oncology Group (RTOG)-guided OAR atlas. This DL model supports the segmentation of six required and eight optional OARs guided by the NRG-RTOG 1106 trial, providing precise and reproducible OARs contouring that are ready to be used in radiotherapy practice. Following the OAR contouring recommendation of the NRG-RTOG 1106 trial, we collected and curated three private datasets and two public datasets, comprising a total of 531 patients with partially annotated thoracic OARs. These partially annotated datasets were utilized to develop DeepOAR, which consisted of a shared encoder and 14 separate decoders, with each decoder dedicated to one specific OAR. For model training, we utilized all patients from the two public datasets and 75% of the patients from the private datasets. We reserved the remaining 25% of the private datasets for independent testing. A multi-user study involving 21 radiation oncologists was conducted on 40 randomly selected patients from the independent testing dataset to evaluate the clinical applicability of DeepOAR. The Dice coefficient score (DSC) and average surface distance (ASD) were computed to evaluate the quantitative delineation performance of the model. DeepOAR outperformed nnUNet (the benchmark medical segmentation model) across all 14 OARs, achieving mean DSC and ASD values of 88.4% and 1.0 mm, respectively, in the independent testing set. Multi-user validation demonstrated that 89.7% of DeepOAR-generated OARs were clinically acceptable or required only minor revisions. A comparison using two randomly selected patients showed that the delineation variability of DeepOAR was significantly smaller than the inter-user variation among radiation oncologists. Human editing of DeepOAR’s predictions could further improve OAR delineation accuracy by an average of 3% increase in DSC and 40% reduction in ASD while significantly reducing the workload of radiation oncologists for contouring 14 thoracic OARs by an average of 77.0%. We developed DeepOAR, a DL-based unified contouring model trained using multiple partially labeled datasets, to delineate a comprehensive set of 14 thoracic OARs following the RTOG-guided OAR atlas. Both qualitative and quantitative results demonstrated the strong clinical applicability of DeepOAR for the OAR delineation process in thoracic cancer radiotherapy workflows, along with improved efficiency, comprehensiveness, and quality.