Sigmoid-based supervised contrastive learning with von mises–fisher similarity and memory banks for visual recognition

Abstract Fine-grained image classification is challenging due to subtle inter-class differences and large intra-class variations. Recent sigmoid-based supervised contrastive methods such as SCS-SupCon mitigate negative-sample dilution through adaptive pairwise decision boundaries, but remain limited by the weak angular sensitivity of cosine similarity, batch-limited negative sampling, and unstable decision boundaries. We propose MM-SupCon, which augments the sigmoid-based pairwise contrastive framework with (i) von Mises–Fisher (vMF) similarity with an adjustable concentration parameter $$\kappa$$ , and (ii) a FIFO Memory Bank of historical embeddings that enriches the pairwise reference pool. A gradient analysis shows that vMF similarity amplifies gradients by a factor approaching $$1+2\kappa$$ in the small-angle regime critical for fine-grained discrimination, and a feature-drift bound explains why a short memory ( $$K \le 3$$ ) remains well aligned under cosine annealing. Across five primary benchmarks and one supplementary small-data setting, with diverse CNN and Transformer backbones including DINO-v3 distilled encoders, MM-SupCon consistently outperforms SCS-SupCon and closely related supervised contrastive baselines, with multi-seed variance estimates and corrected significance tests supporting the gains. Boundary-stability and feature-staleness measurements, together with a new style linear-probe metric, verify the underlying mechanisms and the improved common/style disentanglement. Moreover, by replacing multi-view augmentation with single-view encoding and Memory Bank compensation, MM-SupCon reduces training time by approximately $$2\times$$ compared to multi-view baselines while achieving superior accuracy.