FCA-HLP: Multilayer Feature Cross-Activation Network With High- and Low-Level Prototypes for Few-Shot Segmentation

Few-shot segmentation (FSS) strives to segment novel categories using merely a limited number of labeled images. Current prototype learning and correlation learning approaches struggle to effectively harness both high- and low-level information from support and query images, leading to suboptimal segmentation results. In this work, we propose a multi-layer <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</u>eature <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</u>ross-<underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</u>ctivation (FCA) network with <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">h</u>igh- and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</u>ow-level <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</u>rototypes (HLP), which fully utilizes support and query information from both features and prototypes perspectives. Specifically, for the FCA module, we design a simple activation method that uses all the pixel-level support foreground features to activate query features, thereby obtaining activation maps without losing pixel-level detail information of support features. For the HLP module, we combine image-level prototypes with pixel-level prototypes to fully utilize high-level category information and low-level attribute information of support features. Besides, the prototype generation method integrates query information, which further enables the prototypes to better match target query features, especially when there are great differences between query and support images. Extensive experiments on PASCAL-5<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>i</i></sup> and COCO-20<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><i>i</i></sup> under 1-shot and 5-shot settings validate the effectiveness of our FCA-HLP. Our method establishes new state-of-the-art performance. Additionally, we analyze the performance of the multi-layer feature cross-activation network in the absence of the HLP module. The results indicate that even without prototypes, the FCA module can still deliver strong performance. Codes are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/Jiaguang-NEU/FCA-HLP</uri>.