Tool pose estimation in cochlear implant surgery using monocular microscopy

Traditional cochlear implant (CI) surgery is typically performed without tool tracking or detailed preoperative planning using monocular surgical microscopy. We introduce a novel computer vision-based tool tracking method to facilitate augmented reality CI surgery. Meta's Segment Anything Model 2 (SAM 2) is used for 2D segmentation of the CI insertion tool, which is subsequently used in a 3D to 2D registration method that registers a 3D model of the CI insertion tool with video frames using a gradient descent optimization strategy that leverages differentiable rendering. This registration facilitates accurate pose estimation. A validation study was performed on both synthetic ground truth datasets and real surgical datasets. Results show that the method achieves a mean absolute rotation error of less than 1 degree and a translation error of less than 1 mm. These findings demonstrate the method’s potential for accurately localizing and estimating the pose of the surgical tool for augmented reality-assisted CI surgery.