Hyp-OC: Exploiting Hyperbolic Distance Enhances Open-World Contrastive Learning

Open-world contrastive learning aims to learn a compact representational space for known and novel classes. The task is currently dominated by Euclidean and spherical embeddings so that the decisions about class belongings are made using Euclidean distances, or spherical geodesic distances. However, we believe there exists an inherent hierarchical relationship among natural data, which previous methods have failed to capture. We propose Hyp-OC, a new hyperbolic-based model that use the distance in Poincaré ball to distinguish novel classes from unlabeled data and to cluster for open-world contrastive learning. The key to Hyp-OC lies in a vision transformer, which maps output embeddings to hyperbolic space. Through extensive experimentation, we validated the effectiveness of hyperbolic space embeddings in enhancing the performance of open-world contrastive learning. On the ImageNet dataset, Hyp-OC significantly outperforms the current best method by 5.2% and 3.6% on novel and overall classification accuracy, respectively.