Multimodal Remote Sensing Image Registration via Modality Perception and Self-Supervised Position Estimation

Multi-modal remote sensing images registration ensures that images from different sensors or modalities are spatial and informational consistent for effective comparison and analysis. However, due to the non-linear modality gaps that exist between images, making it difficult to focus only on the spatial position differences of the images and ignore the modality gaps. In this paper, to address this issue, we propose a new framework for Multi-Modal remote sensing image Registration, named MMRNet. The proposed framework comprises the following main aspects. First, a novel self-supervised Positional Misalignment Estimator (PME) is designed for multi-modal image registration. PME is able to efficiently overcome the modality gaps and learn the positional differences between multi-modal images more reliably, optimizing the registration loss by minimizing the positional differences directly. Then, a new paradigm of modality translation, termed Modality Perception Module (MPM), is introduced to effectively learn modality gaps and perform modality translation in the case of positional misalignment. Finally, we further design the modality perception guidance loss to supervise the modality translation task, which can encourage the fidelity of the generated pseudo-modality images. Our registration network integrates both rigid registration model and non-rigid registration model. Experimental results demonstrate that the proposed registration framework can obtain obviously superior performance in both rigid and non-rigid image registration tasks on optical-SAR data, optical-map data and optical-infrared data. The code and relevant dataset will be made publicly available at https://github.com/Ahuer-Lei/MMRNet.