Bi-Temporal Feature Relational Distillation for On-Board Lightweight Change Detection in Remote Sensing Imagery

Deep learning models for remote sensing change detection (CD) require significant resources, challenging real-time applications on spaceborne devices. Knowledge distillation (KD) technology is a potential solution that balances model size and CD performance. However, existing KD methods struggle to effectively transfer the teacher's ability to respond to changed landcovers, and also face challenges in assisting the student model in detecting changes in landcovers with subtle visual characteristics. To overcome these limitations, a bi-temporal feature relational distillation (FRD) framework is proposed. The FRD framework includes two designed distillation components: bi-temporal feature distance distillation (BFDD) and contrastive cluster representation distillation (CCRD). BFDD guides the student model to learn the relationship between bi-temporal feature categories from the teacher model and introduces the iteration-wise random selection and bilinear interpolation strategy to solve feature mismatch problem between the teacher and the student. CCRD describes the pixel-level and clusterlevel feature distributions through the designed contrastive cluster (CC) module, guiding the student model to align the change/nonchange features with the teacher model, so as to alleviate the confusion of change/non-change features. Extensive experiments and analyses have confirmed that the proposed FRD framework can develop lightweight models capable of achieving performance comparable to large models, thereby enhancing suitability for realtime CD.