Geo-Aware Contrastive Selection and Weighted Loss for Multimodal Self-Supervised Learning

Urban scene classification, as a critical task in urban planning, has garnered significant attention in the field of computer vision in recent years. Despite the abundance of raw data, existing supervised learning methods rely heavily on large-scale and high-quality labeled datasets. The potential geographic or contextual information in the images is often overlooked, leading to underutilization of available data. To address this issue, a novel self-supervised learning (SSL) framework is proposed in this paper, which integrates geographic location as auxiliary information to enhance SSL performance. A weighted fusion loss module is also designed to effectively attract positive samples while repelling negative samples, thereby enhancing the discriminative ability of the model. Additionally, to mitigate potential negative effects during the early stages of training, a progressive learning strategy is introduced to gradually stabilize the training process. Experimental results demonstrate that the proposed method significantly outperforms direct supervised learning approaches in urban scene classification. It also exhibits an average performance improvement of 6% compared to models pretrained on ImageNet. Furthermore, it achieves state-of-the-art (SOTA) results in self-supervised pretraining.