BAFNet: A Few-Shot Segmentation Algorithm Based on Two-Stage Backbone Network Adaptation Fine-Tuning via Meta-Learning

The objective of few-shot segmentation is to segment novel categories given only a few annotated support images. Current FSS methods typically rely on pretrained backbone networks while often overlooking the inherent discrepancy between pretraining tasks and downstream segmentation tasks. This oversight renders the models susceptible to noise interference and hinders rapid generalization to novel categories. To address these limitations, we propose BAFNet, a novel few-shot segmentation algorithm based on two-stage backbone adaptive fine-tuning. Our approach incorporates a Feature Activation Adapter module into the backbone network, which operates through similarity feature enhancement and low-dimensional adaptive learning. Building upon this foundation, we develop an adapter-based fine-tuning strategy for the training phase that enhances the backbone network’s capacity for extracting category-relevant features while optimizing similarity representation of the extracted features. Additionally, we introduce a support set-driven, in-episode, online fine-tuning strategy for the testing phase, which leverages data augmentation to generate pseudo-query sets for supervised fine-tuning optimization. Through comprehensive quantitative and qualitative experiments conducted on PASCAL-5i, COCO-20i, and the industrial MT Defect Dataset, our results demonstrate that the proposed BAFNet model achieves state-of-the-art few-shot segmentation performance while utilizing the minimal number of trainable parameters. Our method obtains superior performance for both the mean intersection over union and foreground-background intersection over union evaluation metrics, exhibiting remarkable applicability for both general images in complex scenes and industrial defect segmentation under few-shot conditions.