Calibration of Augmented Reality Headset with External Tracking System Using AX=YB

In Augmented Reality, a robust virtual-to-real calibration that aligns the virtual and real spaces is crucial to ensure accurate overlays. Inspired by popular methods in robotics, we propose establishing the virtual-to-real calibration as a hand-eye/robot-world calibration problem using an $A X=Y B$ formulation. This formulation uses both the self-localization of a head-mounted display and the tracking functionality of an external tracking system. Additional techniques are also provided to address the data synchronization issue between the two measurement systems. To further improve the results, we integrate a post-acquisition outlier filter based on the $A X-Y B$ Frobenius norm. Improvements resulting from the filter were first validated by simulation. For the assessment of the complete pipeline, both subjective evaluation based on human perception and objective evaluation based on computer vision methods were used. The results show that the proposed method is accurate and noise-resistant. With only 100 measurement samples collected, the overlay of a tracked object at the farthest distance (1300 mm) in front of the tracking system has an average rotation error of $1.77 \pm 0.54^{\circ}$ and an average translation error of $4.82 \pm 1.71 \mathrm{~mm}$.