Semantic-edge learning for off-road scene segmentation: a clip-guided SAM framework with refinement blocks

This study proposes an off-road scene segmentation method for complex unstructured environments, constructing a novel multimodal road comprehension framework through synergistic integration of CLIP semantic guidance, SAM-based foundational segmentation, and a Semantic-Aware Refinement Block (SARB). To address prevalent challenges of blurred edges and semantic ambiguity in off-road scenarios, we innovatively design a two-stage refinement mechanism: first establishing semantic constraint spaces through the CLIP visual-language model combined with SAM's prompt-driven segmentation, followed by SARB-enabled multiscale fusion of semantic-geometric features. A Qwen-VL-based multimodal prompt engineering framework is developed to enhance scene interpretation through structured textual descriptions. Experimental results on ORFD and Rellis3D benchmark datasets demonstrate the method's breakthroughs in core metrics including accuracy (3.2% improvement), outperforming conventional segmentation approaches in environmental adaptability. This technology provides an innovative solution for environmental perception in unmanned systems operating within complex scenarios such as military reconnaissance and disaster rescue.