Lightweight dual-domain token learning method for hyperspectral image classification

Convolution neural network (CNN) and transformer-based methods have achieved significant results in hyperspectral image (HSI) classification. These works tend to improve the effectiveness of discrimination feature extraction exclusively in the image domain. However, the excellent feature representation ability in the frequency domain is rarely utilized. Hence, in this paper, a lightweight dual-domain token learning method based on CNN-transformer, aiming to effectively utilize information from both image and frequency domains, is proposed. Specifically, first, we enhance the multi-head self-attention mechanism in the image domain to enhance the mining of spectral correlations across the adjacent spectrum. Second, considering that a single value in the frequency domain has an impact on the values of the entire frequency domain, Fourier token learning is designed to ensure the global information of the frequency domain. Finally, the lightweight network achieves state-of-the-art classification performance with an acceptable computational cost.