Joint Multi-Scale Spatial-Spectral-Textual Features for Cross-Scene Hyperspectral Image Classification

Different from single-source hyperspectral image classification (HSIC), cross-scene HSIC aims to learn invariant features of the source data and achieve good classification results on the target data, which is more challenging. Inspired by contrastive language-image pre-training (CLIP), some studies have used fixed textual knowledge as the prior information to attain domain generalization (DG). However, existing studies have considered the multi-granularity of text while neglecting the multi-scale characteristics of the ground objects in the remote sensing images. Therefore, in this article, a multi-scale spatial-spectral feature extraction module is used as the image encoder of the network based on CLIP for cross-scene HSIC. In particular, the 3-D residual convolutional modules of different kernel sizes are used to extract spatial-spectral features of various scales. A set of learnable weights is used for the dynamic fusion of features at different scales. Then, the multi-scale spatial-spectral feature is input into a multi-scale dilated convolution block to learn the effective features and dynamically fuse them for better classification. Meanwhile, the multi-scale spatial-spectral feature aligns with multi-grained textual features for DG based on the supervised contrastive learning-based approach. The effectiveness of the proposed method is validated on the Houston dataset and Pavia dataset.