DMTF-GAN: A Dual-Branch Mapping GAN With Transformer for Few-Shot Hyperspectral Image Classification

Few-shot hyperspectral image classification (HSIC) is a widely investigated area, and generative adversarial network (GAN)-based methods can achieve good performance. However, most existing GAN-based HSIC methods still face several challenges, such as difficulty in modeling long-range dependencies, the reliability of the generated samples is hard to ensures, and limited feature extraction capability. To work through these concerns, a dual-branch mapping GAN with Transformer for few-shot HSIC, namely, DMTF-GAN, is proposed. The proposed DMTF-GAN method effectively integrates GAN and Transformer into a unified framework, and it includes two main parts: a dual-branch mapping generator module and a cross-layer adaptive fusion (CAF) discriminator module. The constructed dual-branch mapping generator module enhances sample feature representation, improves the quality as well as enriches the diversity of generated samples through noise vector grouping, label embedding, and Transformer-based dual-branch interactions. The CAF discriminator module enhances adaptability by capturing complex relationships in the data via a multilayer Transformer and avoids the loss of key information through CAF, thus improving the discriminative ability. The experimental results indicate that, under few-shot conditions, DMTF-GAN outperforms some recent related state-of-the-art HSIC methods which are based on GAN, for example, it can reach an overall accuracy of 91.32% on the GFYC dataset with only 3 training samples per class.