Segment anything model for multiclass building change detection in remote sensing images

Observing building changes in remote sensing images is crucial for promoting the urbanization process. Mainstream change detection methods based on convolutional neural networks (CNN) and Transformers have shown promising results. However, the small inter-class variance and large intra-class similarity of buildings limit the generalization ability of models. Additionally, current methods often focus on detecting changes, ignoring implicit semantics related to buildings. To address these challenges, we propose a multi-class building change detection framework based on the segment anything model (SAM) to detect newly built and demolished buildings. This framework utilizes adapted FastSAM to encodes fine-grained features and learn general representations of buildings. Then, a dual-branch mask-aided supervision strategy is designed to incorporate two independent single-temporal building semantics into the change map, thereby enhancing temporal modeling capability of the model. Finally, the lovasz extension is introduced to optimize the supervision of minority classes and improve the quality of segmentation. Experimental results on the proposed multi-class building change detection dataset show that our method achieves the highest Recall (72.93%) and F1 (75.14%), as well as visualization results that are closer to the ground truth, demonstrating significant advantages compared with CNN-based methods and Transformer-based methods.