A Dual-Branch Network of Strip Convolution and Swin Transformer for Multimodal Remote Sensing Image Registration

Multimodal remote sensing image registration aims to achieve effective fusion and analysis of information by accurately aligning image data obtained by different sensors, thereby improving the accuracy and application value of remote sensing data in engineering. However, current advanced registration frameworks are unable to accurately register large-scale rigid distortions, such as rotation or scaling, that occur in multi-source remote sensing images. This paper presents a stable and high-precision end-to-end registration network that incorporates dual-branch feature extraction to address the stringent registration requirements encountered in practical engineering applications. The deep neural network consists of three parts: dual-branch feature extraction, affine parameter regression, and spatial transformation network. In the upper branch of the dual-branch feature extraction module, we designed a combination of multi-scale convolution and Swin Transformer to fully extract features of remote sensing images at different scales and levels to better understand the global structure and context information. In the lower branch, we incorporate strip convolution blocks to capture remote contextual information from various directions in multimodal images. Additionally, we introduce an efficient and lightweight ResNet module to enhance global features. At the same time, we developed a strategy to parallelize various convolution kernels in affine parameter regression networks, aiming to enhance the accuracy of transformation parameters and the robustness of the model. We conducted experiments on panchromatic–multispectral, infrared–optical, and SAR–optical image pairs with large-scale rigid transformations. The experimental results show that our method achieves the best registration effect.