Semantic Uncertainty‐Awared for Semantic Segmentation of Remote Sensing Images

ABSTRACT Remote sensing image segmentation is crucial for applications ranging from urban planning to environmental monitoring. However, traditional approaches struggle with the unique challenges of aerial imagery, including complex boundary delineation and intricate spatial relationships. To address these limitations, we introduce the semantic uncertainty‐aware segmentation (SUAS) method, an innovative plug‐and‐play solution designed specifically for remote sensing image analysis. SUAS builds upon the rotated multi‐scale interaction network (RMSIN) architecture and introduces the prompt refinement and uncertainty adjustment module (PRUAM). This novel component transforms original textual prompts into semantic uncertainty‐aware descriptions, particularly focusing on the ambiguous boundaries prevalent in remote sensing imagery. By incorporating semantic uncertainty, SUAS directly tackles the inherent complexities in boundary delineation, enabling more refined segmentations. Experimental results demonstrate SUAS's effectiveness, showing improvements over existing methods across multiple metrics. SUAS achieves consistent enhancements in mean intersection‐over‐union (mIoU) and precision at various thresholds, with notable performance in handling objects with irregular and complex boundaries—a persistent challenge in aerial imagery analysis. The results indicate that SUAS's plug‐and‐play design, which leverages semantic uncertainty to guide the segmentation task, contributes to improved boundary delineation accuracy in remote sensing image analysis.