TextMCD: A mask-classification change detection network with multimodal vision–language supervision

Change detection in remote sensing leverages temporally distinct imagery to identify alterations in the surface of regions of interest, and holds significant application value in areas such as environmental protection and land use. Existing change detection networks consider pixel-level and mask-level perspectives separately, and have achieved promising results. However, methods based on pixel-level change detection are susceptible to challenges, including the merging of change regions and the omission of small change masks. Mask classification based change detection methods, which rely on query predictions, are more susceptible to false changes. To mitigate these limitations, we propose TextMCD, a mask classification based change detection framework that integrates multimodal vision–language learning. Specifically, our network adaptively obtains potential change masks through queries and further constrains the semantic categories of predicted masks using the text label information of a single mask. Meanwhile, we propose a multimodal decoder based on ground truth mask attention mechanism. During the decoding phase, the ground truth is incorporated into the training process without affecting the validation. By combining the mask attention mechanism, the model simultaneously captures the location, category, and temporal embedding information of the masks, which enhances the semantic consistency of individual masks in the change detection process. To validate our method, we performed before-and-after category labeling of change masks on multiple binary change detection datasets. The quantitative analysis reveals a notable increase in F1 score across various datasets. Qualitative analysis further demonstrates the advantages of our network in addressing issues such as mask adhesion, small mask miss detection, and false changes. Our code is available at https://github.com/WuChaseSea/TextMCD . • Proposes TextMCD, a mask-classification network with vision–language supervision. • Introduces a multimodal decoder to enhance mask semantic consistency. • Mitigates mask adhesion, small mask misdetection, and pseudo-change issues.