Robust 360° Visual Tracking with Dynamic Gnomonic Projection

With the growing demand for human-computer interaction in VR and AR videos, 360° video visual tracking has become increasingly important. Since 360° visual content is represented on a sphere, addressing distortions and discontinuities in these video representations during object tracking remains a significant challenge. In this paper, we propose a general method that can be integrated into any 2D visual tracking system to be applied to 360° videos. The core of our method is a bidirectional conversion between gnomonic domain and equirectangular domain. We first map original equirectangular frames into a 2D plane via dynamic gnomonic projection centered on the target object at the current frame and map the 2D tracking result in the next frame back to the equirectangular domain. With our technique, any 2D visual tracking algorithm can then be lifted to track objects within the 360° frames. We use a dataset in equirectangular representation to demonstrate the effectiveness and advantages of our proposed method.