A multimodal multi-scale transformer for virtual pretreatment patient-specific QA of SBRT using portal-dosimetry fluence maps

We developed a transformer-based multimodal neural network to predict the gamma passing rate (GPR) in stereotactic body radiation therapy (SBRT) patient-specific quality assurance. Using 1265 SBRT beams from two institutions, the model incorporated portal dose prediction fluence maps with beam complexity descriptors such as modulation complexity score and monitor units. A multi-scale visual-textual transformer, integrating a ViT encoder and feedforward network through a fusion head, was compared with state-of-the-art CNNs across nine gamma criteria. Our approach consistently achieved the lowest root mean squared error (RMSE) and mean absolute error (MAE), with values ranging from 0.785% to 4.258% and 0.418% to 3.197%, respectively, and ablation studies highlighted the necessity of multimodal fusion and multi-scale design. These results demonstrate superior predictive accuracy and generalizability, underscoring the potential of transformer-based multimodal learning to enhance treatment optimization and clinical QA efficiency.