Contrastive clustering with self-attention enhanced representation encoding

In recent years, deep clustering has garnered considerable interest due to its capability to combine representation learning and clustering through deep neural networks. However, existing deep contrastive clustering methods often struggle with complex image data, primarily due to their limited capacity to capture global information and the constraints of their loss functions. This paper introduces a novel method, Self-Attention-Based Contrastive Clustering (SABCC), which incorporates a self-attention mechanism to address these challenges. Unlike traditional contrastive clustering methods, SABCC utilizes self-attention to capture global features in image data and performs both instance-level and cluster-level contrastive learning. Specifically, we propose two distinct contrastive heads: an instance-level contrastive head to improve inter-sample differentiation and a cluster-level contrastive head to capture the clustering structure across clusters. Furthermore, we introduce the RINCE loss function, which integrates the training signals from both contrastive heads to enhance overall clustering performance. Experiments on six challenging datasets demonstrate that SABCC achieves superior clustering accuracy and consistency compared to mainstream deep clustering methods, showcasing its robust generalization and practical potential.