Geographical Scenario Knowledge-Informed Graph Structure Attention for Image Segmentation

Deep learning methods, renowned for their ability to discern physical features from images, are frequently used in the semantic segmentation of remote sensing images. However, objects with different functional attributes may exhibit similar physical characteristics, resulting in comparable spectral reflectance and visual features. This issue, known as the “different categories with the same spectra” problem, limits the ability to discriminate between objects, thereby increasing the difficulty of differentiation. Studies based on Euclidean space often struggle to distinguish between objects due to the limited information available. To improve differentiation, additional information—such as neighborhood relationships—needs to be incorporated. The geographical scenario, i.e., the environmental context of the object—which includes crucial neighborhood categories and their spatial relationships, provides spatial relationship information for objects. This information provides an important context for object differentiation and becomes the key to distinguishing these similar objects. Following this idea, geographical scenarios are represented as graphs in graph space, with categories as nodes and adjacency relationships as edges, and a geographical knowledge graph is created based on all the scenarios. We convert the remote sensing images into graphs to match with the geographical scenarios and propose a knowledge-based semantic segmentation network for remote sensing, graph structure attention network (GSAN). In GSAN, a graph structure attention (GSAT) is designed based on the graph kernel. This allows it to discern graph structures corresponding to different geographical scenarios. GSAT serves as a link between the fine-grained visual objects and the coarse-grained semantic knowledge. Experiment results indicate that GSAN outperforms other attention networks in semantic segmentation on our sea and land remote sensing (SLRS) dataset. This demonstrates its advantages in geographical scenario recognition and remote sensing semantic segmentation.