End-to-End 3-D Transformer Semantic Segmentation Architecture for Few-Shot Classification With Hyperspectral Multiscale Spatial–Spectral Attention

The application of hyperspectral image (HSI) has expanded from earth observation in large scenes to medical and industrial detection in small scenes. However, there are still many challenges in high-precision few-shot classification with high spectral similarity of targets. We have designed an end-to-end hyperspectral 3D Transformer (HSTR) semantic segmentation architecture based on multiscale spatial-spectral feature attention. This approach aims to simplify the model design and training processes through the end-to-end strategy while accurately capturing overall context and local detail information of spatial-spectral fusion feature. The HSTR can achieve high-precision HSI classification and transfer learning ability when training with limited small sample size, while enhancing efficiency through the utilization of the independent parallel computing advantage with 3D shifted window strategy. Through three typical HSIs experiments, the evaluation of HSTR is conducted alongside nine other methods to assess its performance, and the advantages of HSTR are analyzed from the perspectives of end-to-end architecture, few-shot classification, spatial-spectral features combination and multiscale features extraction. The HSTR algorithm effectively achieves the full extraction of multiscale features and spatial-spectral features fusion while demonstrating the most superior generalization performance and transfer learning ability in HSI classification tasks with insufficient samples.