Technical Approach for Infrared Tracking for Soft Tissue Navigation with a Holographic Head-Mounted Display and Preclinical Validation

Augmented Reality (AR) has the potential to enhance surgical guidance by superimposing three-dimensional (3D) anatomical information directly onto the patient during surgical procedures. However, the practical implementation of AR encounters significant challenges, particularly in accurately tracking organs that move freely during surgical manipulation. Consequently, reliable organ tracking methods are necessary to maintain precise holographic overlays intraoperatively. Preclinical validation of holographic visualizations regarding accuracy poses additional challenges, requiring experimental protocols for quantitative assessment. This protocol addresses these two challenges: it describes a comprehensive approach for developing AR visualization applications using custom-made infrared markers for real-time organ tracking using a Head-Mounted Display (HMD), and it provides a validation framework leveraging electromagnetic (EM) tracking to validate holographic accuracy in phantom experiments. This work outlines step-by-step guidance for creating patient-specific 3D models from medical imaging, designing and manufacturing custom infrared markers, integrating these markers into an AR application for an HMD, and deploying them for surgical navigation. Additionally, it details a validation procedure by using EM-tracking to quantitatively measure the precision of holographic visualizations in semi-deformable kidney phantoms. Therefore, this protocol both facilitates real-time organ tracking and establishes a preclinical validation methodology. Implementing real-time organ tracking could enhance surgical guidance for free-moving organs by accurately overlaying holograms, potentially leading to improved surgical accuracy and better patient outcomes.