Adaptive Fine-Tuning and Feature Enhancement of SAM for Remote Sensing Instance Segmentation

Remote sensing image instance segmentation aims to accurately separate independent objects and automatically identify land attributes. Recent advancements in large models have propelled self-supervised learning, especially with the segment anything model (SAM). Although SAM can generate instance segmentation masks without additional training, its application to remote sensing images still faces challenges, such as a severe reliance on manual prior prompts, inaccuracies in detail segmentation, and insufficient generalization capability. Therefore, to enhance the generalization capability of SAM in the field of remote sensing images, we propose a SAM-based fine-tuning model called AFFE-SAM. This model combines adaptive low-rank (AdaLoRA) and variable adapter (MiMi-Adapter) fine-tuning methods. Additionally, it improves mask quality in the decoder by employing adaptive multi-scale feature enhancement. We conducted experiments on two datasets, obtaining mAP scores of 66.5 for the NWPU VHR-10 dataset and 67.1 for the SSDD dataset. The evaluation results validate the effectiveness of our method.