Multimodal Hierarchical Attention Framework for Efficient Weakly Supervised Few‐Shot Segmentation Under <scp>SAGIN</scp> Environment

ABSTRACT In the evolving landscape of the Space Air Ground‐Integrated Network (SAGIN), addressing the challenges of limited labeled data and the need for adaptable models is crucial for effective data processing across diverse and heterogeneous sources. Weakly supervised few‐shot semantic segmentation (WSFSS), aims to segment the unseen targets by solely relying on support images with class‐level annotations, offering a robust solution to the complexities inherent in SAGIN environments. Existing works usually generate pseudo masks for training images, and then convert WSFSS to the plain few‐shot semantic segmentation task. However, these rough pseudo masks, almost always, contain background noise due to imprecise object localization during mask generation, which thus leads to undesirable segmentation results. To mitigate the above challenge, we inject implicit text supervision into WSFSS, and propose an efficient text‐guided hierarchical attention (ETHA) framework to explicitly alleviate the mask noise issue. Specifically, we first propose a cross‐modal interaction attention module to capture comprehensive object guidance from text embeddings and refocus the model's attention toward the area of the focused object. In addition, we propose a lightweight dual visual cross‐attention module to efficiently aggregate the contextual information among each branch and common object clues from both branches, which provides enhanced visual features to facilitate the cross‐modal information interaction. Based on single‐scale features, ETHA has established new state‐of‐the‐art results on the golden WSFSS datasets, that is, PASCAL‐ and COCO‐. These results highlight ETHA's potential for improving accessibility and efficiency for robust applications in SAGIN environment.