An Occlusion-Robust Optical-Inertial Tracking Solution for Augmented Reality Headsets

Augmented Reality (AR) is a technology that is increasingly used to guide manual tasks and aid the user in enabling vision that would not be possible with the naked eye. For example, in the surgical field, this aid consists of visualising occluded anatomical areas or other additional information along the direction of view. The present work aims to develop a sensor fusion algorithm for an AR system based on hybrid optical-inertial tracking incorporating head kinematics. The developed technology ensures accurate and robust tracking, which can cope with partial and total occlusions of optical markers by implementing a method that integrates data from video cameras with data from a Magnetic-Inertial Measurement Unit (MIMU) sensor.