CLIFF: A Multi-Modal Remote Sensing Model for Geological Hazard Monitoring Based on Bitemporal UAV Images

UAV-based remote sensing excels in rapid response, high timeliness, simple operation, and high degrees of automation, and has been widely applied for geological hazard monitoring. Deep learning methods based on unitemporal UAV images can only analyze the static appearance of a scene, while bitemporal change detection can capture the dynamic evolution of hazards; however, due to diverse geological landforms and topography, environmental noises such as vegetation cover, and dynamic weather conditions, change detection of geological hazards from UAV images based on traditional deep learning technology is not always effective. Therefore, there is an urgent need to utilize large vision-language models (LVLMs) to further improve the accuracy and robustness of the change detection model. Motivated by this, this paper proposes a novel remote sensing model for geological hazard monitoring, referred to as CLIFF (CLIP-BIT-EfficientNet), based on the multi-modal LVLM Contrastive Language–Image Pre-training (CLIP), the change detection network Bitemporal Image Transformer (BIT), and the classification network EfficientNet, along with corresponding datasets and model fine-tuning strategies. The proposed transfer fusion module bridges the CLIFF and BIT networks by aligning their feature distributions and dimensions, allowing the general knowledge of the LVLM and the task-specific knowledge of the learnable branch to reinforce each other. Furthermore, this integrated pipeline addresses the scarcity of labeled hazard data by allowing the BIT to train on larger public datasets, while fine-tuning EfficientNet on smaller hazard-classification datasets within the change area, making the approach more efficient and reliable than direct classification methods. Experimental results show that the proposed CLIFF algorithm outperforms state-of-the-art deep learning algorithms such as LightCDNet and ChangeFormer, with an IoU of 75.74% and an F1 score of 0.8689 for change detection. Meanwhile, CLIFF has an overall accuracy rate of 86.89% in identifying geological hazards along gas pipelines, such as crude oil spills, collapses, landslides, and floods, with per-class accuracies of 87.32% and 86.17% for crude oil spills and landslides, respectively.