Insights of an AI agent via analysis of prediction errors: a case study of fluence map prediction for radiation therapy planning

<em>Purpose. </em>We have previously reported an artificial intelligence (AI) agent that automatically generates intensity-modulated radiation therapy (IMRT) plans via fluence map prediction, by-passing inverse planning. This AI agent achieved clinically comparable quality for prostate cases, but its performance on head-and-neck patients leaves room for improvement. This study aims to collect insights of the deep-learning-based (DL-based) fluence map prediction model by systematically analyzing its prediction errors. <em>Methods. </em>From the modeling perspective, the DL model's output is the fluence maps of IMRT plans. However, from the clinical planning perspective, the plan quality evaluation should be based on the clinical dosimetric criteria such as dose-volume histograms. To account for the complex and non-intuitive relationships between fluence map prediction errors and the corresponding dose distribution changes, we propose a novel error analysis approach that systematically examines plan dosimetric changes that are induced by varying amounts of fluence prediction errors. We investigated four decomposition modes of model prediction errors. The two spatial domain decompositions are based on fluence intensity and fluence gradient. The two frequency domain decompositions are based on Fourier-space banded frequency rings and Fourier-space truncated low-frequency disks. The decomposed error was analyzed for its impact on the resulting plans' dosimetric metrics. The analysis was conducted on 15 test cases spared from the 200 training and 16 validation cases used to train the model. <em>Results. </em>Most planning target volume metrics were significantly correlated with most error decompositions. The Fourier space disk radii had the largest Spearman's coefficients. The low-frequency region within a disk of &sim;20% Fourier space contained most of errors that impact overall plan quality. <em>Conclusions. </em>This study demonstrates the feasibility of using fluence map prediction error analysis to understand the AI agent's performance. Such insights will help fine-tune the DL models in architecture design and loss function selection.