Dual domain feature extraction and adaptive spectral-spatial feature fusion network for hyperspectral image classification

Abstract Although some progress has been made in hyperspectral image (HSI) classification, it still faces many challenges due to limited training samples, insufficient fusion of spectral and spatial information, and consumption of computing resources. In order to effectively address the above problems, this paper proposes a novel combination of dual domain feature extraction and adaptive spectral-spatial feature fusion (DDFE-ASFS), which fully extracts global and local spectral-spatial features and deep high-level semantic features. Firstly, a dual domain feature extraction (DDFE) module is proposed by integrating deep CNNs, fast Fourier transform (FFT) and inverse fast Fourier transform (IFFT), which can fully characterize local and global spectral-spatial and frequency features. Secondly, an efficient adaptive spectral-spatial fusion (EASSF) module is designed to capture the dependency between cross-views by using the attention mechanism while maintaining the consistency of spectral and spatial features. Then, two convolution layers are used to further optimize the features, and pixel-attention and residual path are combined to achieve dynamic fusion of spectral and spatial features. Finally, the spectral graph context optimizer (SGCO) is used to model the long-range dependency relationship, and improve the classification efficiency and accuracy. Extensive evaluations on four popular HSIs show that, with 10% of the training samples, the proposed method reaches 99.57% average accuracy on the Houston2013 dataset, 99.80% on the Pavia University dataset, 99.85% on the WHU-Hi-HanChuan dataset, and 99.70% on the WHU-Hi-HongHu dataset, superior to some existing advanced technologies.