TAP-Track: Generalizable Spacecraft Pose Tracking by Tracking Any Points

Recent learning-based spacecraft pose tracking methods have demonstrated impressive improvement in estimation accuracy and potential scalability to complex space environment. However, most of them still rely on the detection of discriminative keypoints on a known 3D model, limiting the generalization to unknown spacecraft. To this end, we propose, to the best of our knowledge, the first generalizable spacecraft pose tracking method. Instead of requiring a known model, we only assume the existence of at least one planar structure, e.g. solar panels, which holds for most satellites in general scenes. Additionally, the proposed method tracks any points on the plane across multi-frame followed by a re-projection error minimization, rather than detecting keypoints between image pairs, allowing robust capture of “long-term” temporal information among frames even for textureless surfaces without sufficient keypoints. Moreover, we also constructed the first large-scale dataset G-SPET for generalizable spacecraft pose estimation and tracking. It covers 174 satellites with diversity structures and rich annotations, increasing the number of targets in previous datasets by almost two orders of magnitude. Extensive evaluations on the proposed dataset have demonstrated the superiority of our method over state-of-the-art methods. The code and dataset will be made publicly available soon.