‘See’ through the surface: surface-derived three-dimensional AI-driven real-time imaging solution for intra-treatment image guidance

Respiratory motion is a long-standing challenge for lung stereotactic body radiotherapy (SBRT), particularly for centrally located lung tumors where increased toxicity demands more precise motion management during treatment. Current two-dimensional imaging approaches are insufficient for 3D tumor deformable motion tracking. In this study, we developed and evaluated a Surface-derived Three-dimensional (3D) AI-driven Real-time (STAR) imaging system by transforming a surface imaging system into a 3D real-time imaging solution. STAR integrated two key components: (1) prior-model-free spatiotemporal implicit neural representation (PMF-STINR): a machine-learning sub-system for pretreatment dynamic cone-beam CT (CBCT) reconstruction and motion modeling; and (2) surface-to-deformation network (Surf2DefNet): a deep-learning model that correlates intra-treatment body surface images with internal 3D anatomy and motion fields, trained based on the dynamic CBCT and motion model output of PMF-STINR. Specifically, PMF-STINR reconstructs a reference CBCT and solves an eigenvector-based motion model from a pretreatment CBCT, while Surf2DefNet predicts the motion eigen-vector weightings from surface images, enabling it to infer real-time CBCTs and motion vector fields (MVFs) using intra-treatment surface maps acquired later. We evaluated STAR imaging using both a digital extended cardiac torso (XCAT) phantom with regular and irregular motion patterns and ten patient datasets. The relative error (RE), center-of-mass error (COME), 95th percentile Hausdorff distance (HD95), Dice coefficients (DICE), and Pearson correlation coefficient (PCC) metrics between STAR images and the 'GT' were evaluated. For the XCAT phantom and the ten patients, the mean COME values are within 1 mm for all but one patient (1.3 mm). The RE values were consistently low, and the DICE and PCC values exceeded 0.89 in all cases. The HD95 are all within 2 mm except for one patient (2. 78 mm). These results demonstrate that the STAR imaging system can achieve accurate spatiotemporal reconstructions from surface images, providing CBCTs and MVFs for intra-treatment real-time image-guidance, and has the potential to improve safety and efficacy of SBRT for lung cancer.