A Remote Sensing Change Detection Network Using Visual-Prompt Enhanced CLIP

Remote Sensing Change Detection (RSCD) plays a crucial role in various earth observation tasks. Recently, deep learning-based methods have been widely employed for CD due to their exceptional performance. Although, existing approaches can detect obviously changed regions easily, they suffer from difficulties in dealing with pseudo-changes caused by lighting condition changes, season changes and complex land cover conditions. To tackle this challenge, we propose a CD network using visual-prompt enhanced CLIP (CVNet), which incorporates the foundation model CLIP into the RSCD task to leverage its semantic information for identifying pseudochange regions. Specifically, we use CLIP visual encoder with transformer-structure to enhance single-time features extracted by ResNet. To ensure effective transfer ability for downstream tasks while considering computational cost, we fine-tune CLIP using a visual prompt. In addition, to efficiently enhance features extracted by ResNet, we design a CLIP-guided feature refinement (CGFR) module that adaptively integrates both types of features. Furthermore, a transformer encoder structure is introduced to get change information for dual-time images and a transformer decoder is introduced to propagate change information back. To test the performance of our proposed model, we conduct experiments on two datasets, including LEVIR-CD and WHU-CD. The experimental results show that our model can outperform several state-of-the-art models on both datasets. The code is available at https://github.com/Hyper-Baller/CVNet.