Hyperspectral Unmixing via Multi-scale Representation by CNN-BiLSTM and Transformer Network

Hyperspectral unmixing (HU) is a powerful technique to solve the challenge associated with mixed pixels that widely exist in hyperspectral images (HSIs). HU can obtain the pure signature of ground objects (endmembers) and their constituent proportion (abundances) within mixed pixels. Recently, deep learning techniques, with their powerful feature extraction capabilities, have significantly advanced the development of new HU technologies. A HU network based on an autoencoder, which combines a multi-branch convolutional neural network (CNN), a bidirectional long and short-term memory (BiLSTM) network, and transformers is proposed to enhance the spectral and spatial feature extraction ability of the network in this paper. Given the complex distribution of ground objects observed in hyperspectral remote sensing images, we employ a multi-branch CNN and a BiLSTM to extract multi-scale spatial features and spectral features respectively to address the issue that the transformer is not sensitive to low-level features. Then, the joint features are input into the transformer encoder to extract non-local features. Finally, the decoder reconstructs the HSI, estimating both endmembers and abundances. The efficacy of the proposed method is validated through one synthetic hyperspectral dataset, in addition to three real hyperspectral datasets: Samson, Jasper, and Apex.