Text-to-Image Activation for Open-Vocabulary Semantic Segmentation in Remote Sensing

Open-vocabulary semantic segmentation in remote sensing aims to recognize arbitrary object categories from satellite imageries beyond a fixed label set, but its progress is constrained by the lack of fine-grained pixel-level supervision during training. Text-based weak supervision offers a scalable alternative by generating pseudo-labels from image-level annotations, yet in remote sensing scenarios it faces twofold challenges:1) incompleteness and unreliability of pseudo-labels, and 2) segmentation learning degradation induced by label noise. To address these issues, we propose a framework named Text-to-Image Activation for Open-Vocabulary Semantic Segmentation (T2ASeg), tailored for the spatial and semantic characteristics of satellite imagery. T2ASeg integrates two key modules to enhance pseudo-label quality and segmentation robustness. The Text-guided Adaptive Activation Module (TAAM) generates scale-aware and semantically consistent class activation maps, serving as reliable spatial and semantic cues for pseudo-label generation. The Multi-Scale Structure and Semantic Enhancement Module (MSSEM) combines structural modeling with class-aware semantic guidance, enabling the model to better capture fine-grained semantics and spatial structures under noisy supervision. By synergizing these modules, T2ASeg delivers more accurate and complete segmentation maps, even in the presence of weak and noisy labels. Extensive experiments on multiple remote sensing benchmarks show that T2ASeg outperforms state-of-the-art methods in both accuracy and boundary quality, demonstrating the value of combining textual guidance with adaptive spatial modeling for open-vocabulary segmentation. Code is available at github.com/WyneeWang/T2ASeg.

Authors

Institutions

Publication Details

Journal
IEEE Transactions on Geoscience and Remote Sensing
Published
2025-01-01
DOI
https://doi.org/10.1109/tgrs.2025.3619504
Citations
1
Primary Topic
Image Retrieval and Classification Techniques
Type
article
Field-Weighted Citation Impact
1.28
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Text-to-Image Activation for Open-Vocabulary Semantic Segmentation in Remote Sensing

Liang Xiao, Wei He, Aoran Xiao, Wenzhen Wang et al.
1 citations
IEEE Transactions on Geoscience and Remote Sensing
Image Retrieval and Classification Techniques
1.28
article

Text-to-Image Activation for Open-Vocabulary Semantic Segmentation in Remote Sensing

Liang Xiao, Wei He, Aoran Xiao, Wenzhen Wang, Hongyuan Zhu
article en
1 citations

Abstract

Open-vocabulary semantic segmentation in remote sensing aims to recognize arbitrary object categories from satellite imageries beyond a fixed label set, but its progress is constrained by the lack of fine-grained pixel-level supervision during training. Text-based weak supervision offers a scalable alternative by generating pseudo-labels from image-level annotations, yet in remote sensing scenarios it faces twofold challenges:1) incompleteness and unreliability of pseudo-labels, and 2) segmentation learning degradation induced by label noise. To address these issues, we propose a framework named Text-to-Image Activation for Open-Vocabulary Semantic Segmentation (T2ASeg), tailored for the spatial and semantic characteristics of satellite imagery. T2ASeg integrates two key modules to enhance pseudo-label quality and segmentation robustness. The Text-guided Adaptive Activation Module (TAAM) generates scale-aware and semantically consistent class activation maps, serving as reliable spatial and semantic cues for pseudo-label generation. The Multi-Scale Structure and Semantic Enhancement Module (MSSEM) combines structural modeling with class-aware semantic guidance, enabling the model to better capture fine-grained semantics and spatial structures under noisy supervision. By synergizing these modules, T2ASeg delivers more accurate and complete segmentation maps, even in the presence of weak and noisy labels. Extensive experiments on multiple remote sensing benchmarks show that T2ASeg outperforms state-of-the-art methods in both accuracy and boundary quality, demonstrating the value of combining textual guidance with adaptive spatial modeling for open-vocabulary segmentation. Code is available at github.com/WyneeWang/T2ASeg.

IEEE Transactions on Geoscience and Remote SensingVol. 63
Agency for Science, Technology and Research (SG), Institute for Infocomm Research (SG), Nanjing University of Science and Technology (CN), Wuhan University (CN), State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing (CN), RIKEN Center for Advanced Intelligence Project (JP)
Openalex Percentile: Top 15%
Image Retrieval and Classification Techniques
1.28
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.