Markerless Registration and Visualization of Volumetric Medical Data for Enhanced Surgical Precision Using Head-Mounted Displays

In surgery, accurate localization of critical structures like blood vessels is crucial, yet current 2D visualization methods limit precision and increase cognitive load for surgeons. This paper introduces a markerless registration approach that uses only sensors in a Head-Mounted Display to align CT/MRI data directly onto the patient in a 3D space. Point clouds extracted from CT/MRI data are registered with world captures from the Head-Mounted Display’s time-of-flight camera, allowing for precise visualization. We evaluated the system using physical measurements on phantoms and compared algorithmic registration to manual alignment. The results show a registration error of approx. 4.87±0.05mm, demonstrating the potential of this technique for preoperative planning and intraoperative support.