New insights into automatic treatment planning for cancer radiotherapy using explainable artificial intelligence

<i>Objective.</i>This study aims to uncover the opaque decision-making process of an artificial intelligence (AI) agent for automatic treatment planning.<i>Approach.</i>We examined a previously developed AI agent based on the actor-critic with experience replay (ACER) network, which automatically tunes treatment planning parameters (TPPs) for inverse planning in prostate cancer intensity modulated radiotherapy. We selected multiple checkpoint ACER agents from different stages of training and applied an explainable AI method to analyze the attribution from dose-volume histogram (DVH) inputs to TPP-tuning decisions. We then assessed each agent's planning efficacy and efficiency, and evaluated their policy space and final TPP tuning space. Combining findings from these approaches, we systematically examined how ACER agents generated high-quality treatment plans in response to different DVH inputs.<i>Main results.</i>Attribution analysis revealed that ACER agents progressively learned to identify dose-violation regions from DVH inputs and promote appropriate TPP-tuning actions to mitigate them. Organ-wise similarities between DVH attributions and dose-violation reductions ranged from 0.25 to 0.5 across tested agents. While all agents achieved comparably high final planning scores, their planning efficiency and stability differed. Agents with stronger attribution-violation similarity required fewer tuning steps ( 12-13 vs 22), exhibited a more concentrated TPP-tuning space with lower entropy ( 0.3 vs 0.6), converged on adjusting only a few key TPPs, and showed smaller discrepancies between practical tuning steps and the theoretical steps needed to move from initial values to the final TPP space. Putting together, these findings indicate that high-performing ACER agents can effectively identify dose violations from DVH inputs and employ a global tuning strategy to achieve high-quality treatment planning.<i>Significance.</i>This study demonstrates that the AI agent learns effective TPP-tuning strategies, exhibiting behaviors similar to those of experienced human planners. Improved interpretability of the agent's decision-making process may enhance clinician trust and inspire new strategies for automatic treatment planning.