Hierarchical and Bidirectional Contrastive Learning for Hyperspectral Image Classification

Representing hyperspectral images (HSI) is a complex and challenging task, primarily due to spectral uncertainty. Learnable prototypical contrastive learning is specialized in discriminative instance representation. However, it requires a much low temperature to prevent model collapse, which can hinder the encoder’s ability to capture category relationships. Furthermore, self-supervision at network terminal could obscure specific semantics in hyperfine spectra. In this paper, we propose a hierarchical and bidirectional learnable prototypical contrastive learning method (HiBiCo) for HSI representation. We build learnable prototype dictionaries at shallow and final layers of the network for deep contrastive supervision. Moreover, we introduce reverse contrastive learning under negative sample dominance to address excessively uniform representations caused by traditional positive-dominated contrastive loss in dual-dictionary hierarchical supervision. By allowing deep contrastive supervision and two-way information flow along with the general InfoNCE loss, our approach alleviates the uniform distribution, maintains the latent data manifold, and enables diverse and effective representation. Experiments with linear probing demonstrate the effectiveness of our HiBiCo framework in handling complex scenes, highlighting the potential of self-supervised pretraining for hyperspectral image representation. The code is available at https://github.com/sakurashine/HiBiCo.