Multi-Granularity Aggregation Network for Remote Sensing Few-Shot Segmentation

Few-shot semantic segmentation (FSS) aims to segment a query image using a limited number of densely annotated support images from the same category. Most existing conventional FSS methods are tailored for coping with images from natural scenarios. Unlike natural images, remote sensing images usually have a similar background context among the support and query image pairs, and more severe intraclass inconsistency exists due to overhead shooting views. However, facing such realistic and challenging remote sensing FSS tasks, the existing methods seldom consider these intrinsic characteristics from a unified viewpoint, thus leading to inferior results. To solve the above dilemma, we propose a multi-granularity aggregation network (MGANet) to progressively capture multi-granularity discriminative information, for tackling the remote sensing FSS task. Specifically, MGANet consists of a multi-granularity similarity (MGS) module and an adaptive multiprototype aggregation (AMPA) module. To fully utilize background context, MGS extracts multi-granularity support and query feature maps from the backbone network to calculate a holistic correlation by incorporating the background information. Next, to alleviate the intraclass inconsistency of remote sensing images, AMPA decomposes the support foreground region into mainstay and auxiliary subregions by the guidance of reverse prediction on support features, thus generating three types of prototypes by masked average pooling (MAP) on these paired features and masks. Furthermore, these multiprototypes are collaboratively interacted with the query features to pursue reinforced discriminative features, relying on prototype-aware slot attention (PASA). Extensive experiments on iSAID-<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$5^{i}$ </tex-math></inline-formula> and LoveDA-<inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$2^{i}$ </tex-math></inline-formula> demonstrate well the superiority of the proposed MGANet. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/CVL-hub/MGANet/</uri>.