Reinforcement learning with mechanistic models to optimise radiotherapy and immunotherapy combinations: a proof of concept

<i>Objective.</i>To investigate the use of reinforcement learning (RL) algorithms to optimise complex combination cancer therapies. The RL algorithm investigated the effect of varying the radiotherapy (RT) dose in each fraction when administered in conjunction with the immune checkpoint inhibitors (ICIs) anti-PD-1 and anti-CTLA-4.<i>Approach.</i>Data were available for BALB/c mice inoculated with a syngeneic mesothelioma tumour on the flank, treated with combination RT and ICI with tumour growth subsequently measured. A deep<i>Q</i>-network (DQN) and a double DQN were trained using a mechanistic model fitted to the mesothelioma volumes to simulate the dynamics of the tumour microenvironment. Two reward functions were created for the RL algorithm to optimise: the first only considered tumour cell killing, while the second penalised treatment schedules with higher total RT dose. Comparison with experimental results was via the tumour control probability (TCP).<i>Main Results.</i>All the TCPs obtained with the RL algorithm exceeded the TCPs obtained with the same mechanistic model when only 1 or 2 fractions of RT were administered. However, the baseline schedule of 2 Gy per fraction outperformed the treatment schedules generated by RL.<i>Significance.</i>This study highlights the potential for RL to explore the vast solution space of possible treatment schedules, conceivably at the individual patient level.