A multi stage coarse-to-fine network with semantic similarity for infrared and visible image registration

Multimodal image registration aims to align images within a common geometric space and is essential for tasks like image fusion and segmentation. However, existing methods often focus only on rigid or non-rigid transformations, which limits their effectiveness when both are present. We propose a coarse-to-fine image registration framework, termed SA-CFNet. It performs rigid image registration in the coarse stage and non-rigid image registration in the fine stage. It consists of two key components: the Rigid Transformation Network (RNet) and Semantic-Guided Deformation Network (SGDNet). RNet estimates a 6D transformation matrix that applies rotation and translation to the moving image, yielding a coarsely registered result. To establish reliable correspondences, we use a semantic similarity-based strategy that leverages semantic information from the images. SGDNet refines the registration by estimating a deformation field, allowing the coarse_registration_image to be further adjusted for fine registration. The deformation field is also calculated using semantic similarity. We employ a coarse-to-fine estimation strategy to enhance the accuracy of the deformation field. We conducted experiments on image rigid transformation and non-rigid transformation using datasets such as RoadScene, TNO, and M3FD. To validate the effectiveness for downstream tasks, we designed an experiment on multimodal image fusion. The results show that our method remains robust in handling different rotations and translations for rigid image registration and achieves state-of-the-art performance in image registration tasks.