168 Leveraging the Smarts in Your Phone: An AI-Driven iOS App for Surgical Navigation

INTRODUCTION: Surgical navigation allows for safe, accurate and precise neurosurgery. Utilizing standard iOS devices may allow surgeons to use these tools outside of ORs and in low-resource settings. METHODS: We built an iOS app to provide real-time surgical navigation. The intent was that a standard iPhone or iPad provide a complete surgical navigation experience run entirely on the local mobile device. We trained, tuned, and validated 3 AI models: a semantic segmentation model for brain anatomy, semantic segmentation for faces, and object detection for a custom stylet attachment. GPU programming was used to accelerate on-device real-time, continuous registration and optimized for low power consumption. RESULTS: A UNet was trained on 8-1mm Head CTs resulting in a 98.3% testing and 98.2% validation accuracy using a 50%-50% test/validation split, and segments a thin-cut CT in 3 seconds on an iPhone 12. Point cloud merges of patient anatomy took 4 seconds with an initial depth scan of 30,000 points, and is updated in real-time to achieve a cumulative error following scaling, alignment, and rotation of 1x10-8 cms. Transfer learning powered EVD tracking trained for 1000 epochs resulted in an I/U of 1.0 and varied I/U of 0.98 for our detection model, and runs on Apple’s neural engine with inference times of 800us. A cadaver study is set for June, results submitted to late-breaking abstracts to characterize accuracy. CONCLUSIONS: iOS devices equipped with TrueDepth cameras are capable of providing real-time, continuous surgical navigation for non-immobilized heads. We demonstrate how a combination of AI models can register patient anatomy, identify and track surgical instrumentation, and provide neurosurgeons with an intuitive experience to place EVDs on an iPhone or iPad.