Real-time Volumetric Imaging for Minimally-Invasive Thoracic Interventions: A Deformable Phantom Study

Objectives: This study aimed to evaluate the ability of VoxelMap, a deep learning framework for 3D respiratory motion estimation, to generate volumetric images in real-time from 2D projections acquired for two deformable lung phantoms on a clinical interventional C-arm angiography system. Materials and Methods: VoxelMap was evaluated using the PixelPrint4D and Casper deformable physical thoracic respiratory phantoms. Four-dimensional cone-beam CT (4D CBCT) images acquired on a clinical fixed-room C-arm angiography system were used for network training and volumetric ground truth. Prediction accuracy was evaluated on withheld respiratory phases using structural similarity index measure (SSIM) and peak signalto-noise ratio (PSNR). Model inference time was measured to assess suitability for continuous real-time processing. Deep ensembles were also used to assess model uncertainty. Correlation between ensemble uncertainty and prediction error was evaluated for both phantoms. Results: VoxelMap achieved high prediction accuracy across withheld respiratory phases in both physical phantoms. SSIM values exceeded 0.92 in all experiments, with a higher performance observed for the Casper phantom (0.969-0.973) than for the PixelPrint4D phantom (0.927-0.943). Model inference times were less than 15 ms for both phantoms which is compatible with real-time processing. Ensemble uncertainty was positively correlated with prediction error in both phantoms, indicating that higher uncertainty was associated with less accurate volumetric predictions. Conclusions: Our study demonstrated that VoxelMap generates accurate volumetric deformations of respiratory anatomy from projections acquired on a clinical interventional C-arm system, with computational inference times compatible with continuous processing. By deriving volumetric information directly from two-dimensional fluoroscopic projections, VoxelMap has the potential to augment continuous fluoroscopic guidance with continuously updated volumetric information. Ensemble uncertainty may provide a potentially useful indicator of prediction reliability. Further studies using clinically representative training data and patient studies is required to establish clinical utility. Our code is available at https://github.com/Image-X-Institute/Voxelmap.