Adapting Foundation Features via Cross-View Contrastive Learning for Unseen Object Pose Estimation

Recent advances in visual foundation models, such as DI-NOv2, have shown impressive generalization capabilities, making them a strong baseline for complex tasks like unseen object pose estimation. However, a significant domain gap between real images and rendered templates, combined with limited 3D awareness, hinders their effectiveness in precise correspondence matching and pose estimation. In this paper, we propose a new adaptation method that reduces this domain gap and enhances multi-view consistency through cross-view contrastive learning. Our approach aligns features from real and rendered images by using sampled poses and 2D-2D correspondences while also leveraging advanced augmentations and PCA-based loss computation to improve robustness and efficiency. When applied to the FoundPose model, our method achieves state-of-the-art performance on the BOP benchmark, surpassing previous methods with minimal fine-tuning.