Automated Assessment of Nondosimetric Aspects of Prostate Radiation Therapy Treatment Quality in a Large Plan Database

Purpose: Comprehensive assessment of radiation therapy treatment quality in large-scale multi-institutional contexts remains an outstanding challenge. Human review of retrospective treatment plans is labor intensive and impractical for large data sets. Dosimetric plan quality assessment can be approached with knowledge-based planning, yet a standardized, automated approach is lacking for other aspects of treatment quality, such as target contours, margins, and the use of image guidance with fiducial markers. In this work, we develop novel automated methods for assessing nondosimetric plan quality indicators over large databases of prostate radiation therapy treatments. The purpose of the algorithms is to flag potentially low-quality plans for detailed human review. This study serves as a proof of concept toward scalable quality assessment across multi-institutional databases. Methods and Materials: A DICOM database of 1395 patients and 1671 prostate radiation therapy plans was analyzed. Automated clinical target volume (CTV) and planning target volume (PTV) identification was based on structure set naming conventions. PTV margins were automatically derived by computing mean Euclidean surface distances between the CTV and PTV boundaries. Fiducial marker presence was determined by intensity thresholding and voxel clustering. Finally, target contour outliers were identified by a V-Net autocontouring model. To validate the accuracy of our models, ground-truth quality measurements were recomputed by hand. Results: The models were trained on 1328 patients from our institution and tested on 67 patients from the greater community. PTV identification accuracy was 64/67 (95.5%). Plan type accuracy was 62/67 (92.5%). The models obtained a margin computation agreement of 0.24 ± 0.13 mm and a prescription computation agreement of 5 ± 22 cGy with values manually derived from clinical software. Fiducial marker detection was 67/67 (100%). Autocontouring agreement was 82.4% ± 5.4% on intact prostates and 63.1% ± 11.0% on prostate beds. Statistically significant aggregate differences in PTV margins, fiducial marker usage, and prostate target contours were observed between internal and external plans. These models identified 3/67 (4.5%) plans that were manually reviewed by radiation oncologists and confirmed to be suboptimal. Conclusions: Automated algorithms for quantitative measurements of plan quality were developed, and high accuracy was achieved on external data sets. Our methodology is pertinent to analyzing large radiation therapy databases for quality control. The relatively small external testing cohort limits generalizability, highlighting the need for future multi-institutional validation.