Selective Spectral–Spatial Aggregation Transformer for Hyperspectral and LiDAR Classification

Convolutional neural networks (CNNs) and transformers have achieved excellent classification performances in hyperspectral imagery (HSI) and light detection and ranging (LiDAR) land cover classification. However, for complex land covers, effectively characterizing the contextual information and spectral-spatial interaction features of HSI and LiDAR is crucial for improving classification accuracy. Motivated by this, this letter is dedicated to selective convolutional kernel mechanisms and spectral-spatial interactive transformer feature learning style, proposing a selective spectral-spatial aggregation transformer network, named S2ATNet. A convolution feature selected module (CFSM), which can dynamically capture the contextual features of various land covers, is first utilized in both of HSI and LiDAR branches. Afterward, a cascaded spatial-spectral learning and interactive fusion (CSLIF) block is designed for acquiring the nonlocal spatial-spectral characteristics in an interactive feature learning style. The learned features are fed into the max-average classification head (MACH) to obtain the final classification results. The effectiveness of the proposed S2ATNet is validated on two publicly available datasets. Codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/RSIP-NJUPT/S2ATNet.git</uri>.