Artificial intelligence-assisted spatial correction of augmented reality projections; implementing radiation mapping during radioguided robotic surgery

Abstract Translation of the drop-in gamma probe transformed radioguided robot-assisted prostate cancer (PCa) surgery. This study evaluated whether digitization of radiometric signals could allow conversion into more easier to interpret Augmented Reality radiation mapping (AR RM ) displays. 20 PCa patients were included who underwent robot-assisted sentinel node (SN)- and prostate-specific membrane antigen (PSMA) radioguided resections (24 SNs, 16 PSMA-avid nodes and 1 PSMA-avid primary tumor). Radiometric probe counts/s (dimensionless) synchronized with positional probe data allowed the retrospective creation of dynamic AR RM overlays in two dimensions. Artificial intelligence was used to segment out, and automatically track, surgical instruments and dynamically de-occlude the AR RM displays accordingly. Retrospective scoring by end-users indicated a 72.5% preference for the AR RM displays compared to the currently available radiometric display (bipolar scale; median ≥2; p < 0.001). Thus, indicating that video-feed embedded AR RM visualizations provide a digital means to further advance the surgeon’s perception of intraoperative molecular imaging.