Monocular Vision‐Based Endoscopic Sinus Navigation: A SLAM Driven Approach With CT Integration

Surgical navigation is critical in sinus surgery to enhance the surgeon's spatial awareness and improve precision, particularly around occluded critical structures. While external tracker-based navigation systems exist, vision-based solutions are preferred for being less intrusive and for enabling endoscopic image analysis to assist surgeons. However, monocular endoscopy navigation faces challenges associated with monocular reconstruction and camera pose estimation. This paper presents a proof of concept for monocular vision-based sinus navigation that utilizes only preoperative CT data and the endoscope video stream to navigate the sinus anatomy. We developed a vision-based navigation system that incorporates a SLAM algorithm to estimate the camera pose and reconstruct the 3D surface of the anatomy. Given an initial semi-automated registration, the algorithm maps the SLAM-based trajectories to the CT space while employing the reconstructed point cloud to solve for the scale interactively. The system displays the updates in the CT triplane visualization as SLAM reconstructs the scene and recovers pose information. We tested our system by performing an off-site navigation in ten recorded endoscopic video streaming generated from sequences obtained from eight cadaveric subjects, comparing the vision-based navigation to reference optical tracker pose data and obtaining translation and rotation errors of 3.2 mm and 4.9 degrees, respectively. Additionally, we performed three on-site tests of our system on two different cadaver experiments. Our work evaluates a fully integrated system that closes the loop between image-based reconstruction and CT visualization, and discusses the challenges to address to achieve clinical level surgical navigation.