Augmented Reality Navigation with Optical Coherence Tomography in Robotic Neurosurgery

An important issue commonly faced in neurosurgical procedures is the lack of Augmented Reality (AR) navigation and perception with sub-millimeter accuracy. The most commonly used magnetic resonance imaging (MRI) scans provide planar imaging of brain structures and corresponding three-dimensional reconstruction results. However, the principles of MRI imaging limit the imaging speed of this modality, and the millimeter-scale resolution is not always satisfactory. Recently, an optical coherence tomography (OCT) imaging method has been applied in neurosurgical procedures due to its higher spatial resolution and imaging speed, and has achieved the goal of deep brain imaging using endoscopic lens designs. However, the limited depth of field and cross-sectional imaging plane of OCT imaging technology often confuse novice users and pose difficulties for real-time understanding during surgical procedures. For this purpose, we explored a method for visualizing OCT images in augmented reality (AR) wearable devices. We collect real-time OCT data through the Robot Operating System (ROS) and perform rendering processing. Then, we develop the AR system interface in Unity software and perform real-time visualization of OCT images. Real-time updating and display of OCT images enable users to observe changes in brain structures during surgical procedures, identify tissue structural content in OCT images, and perform more precise surgical operations.