Refining Remote Sensing Image Segmentation Results Based on Vision Foundation Models

Pixel-level classification of remote sensing images is a fundamental task in Earth science-related research. However, current automated models inevitably produce some segmentation errors. In practical applications, manual review and correction are often required. To address the inevitable segmentation errors in automated remote sensing image segmentation, this letter proposes a remote sensing segmentation correction model. This model is based on the segment anything (SAM) model and requires training only a small number of parameters to enable the model to perceive click prompt. The model consists of two main components: one part is responsible for automated segmentation and the other part is dedicated to refine the results of the automated segmentation. Considering that click-based correction is difficult to learn during training, we designed a two-stage training process for the network. Through experiments on two widely used datasets, it was found that the proposed remote sensing image correction network significantly improves mIOU after incorporating the correction process. After applying fewer than six corrective clicks per category, the mIOU is improved by 12.99% in ISPRS Vaihingen dataset and 6.93% in ISPRS Postam dataset.