SAM-GFNet: Generalized Feature Fusion with Hierarchical Network for Hyperspectral Image Semantic Segmentation

Visual foundation models exhibit significant potential for extracting generalized characteristics from imagery, which is crucial for remote sensing image interpretation tasks. However, current approaches underutilize these capabilities for hyperspectral image semantic segmentation. To bridge this gap, this study establishes SAM-derived generalized feature fusion with a hierarchical network for hyperspectral image semantic segmentation (SAM-GFNet), which integrates generalized features with hierarchical task-specific spatial–spectral characteristics for the segmentation task. Specifically, hyperspectral images are partitioned into multiple spectral groups, and SAM-induced generalized features are extracted within each group’s corresponding spectral range. To leverage multi-scale contextual information, the hierarchical hyperspectral image semantic segmentation architecture fuses multi-scale spectral–spatial features with SAM-induced generalized visual characteristics through a hierarchical fusion pattern. Furthermore, an efficient feature fusion module is further developed to adaptively integrate encoded features with decoder-generated characteristics, further enhancing semantic segmentation performance. The proposed semantic segmentation scheme is evaluated on WHU-OHS and WHU-H2SR benchmark datasets, achieving overall accuracies of 79.60% and 86.92% on two datasets, surpassing representative baseline methods by over 2.94% and 1.56%, respectively.