SMAF-net: semantics-guided modality transfer and hierarchical feature fusion for optical-SAR image registration

• The SMAF-Net based on semantic-constrained GAN for optical-SAR registration. • Shared multi-scale feature learning module for feature consistency. • Hierarchical feature aggregation module guided by channel attention. Accurate registration of optical and synthetic aperture radar (SAR) images is critical for effective fusion in remote sensing applications. To address the significant radiometric and geometric differences between these modalities, SMAF-Net, a novel network that integrates semantics-guided modality transfer and hierarchical feature fusion for optical-SAR image registration, is proposed. For modality transfer, a feature-constrained generative adversarial module (SGMT) is used to translate SAR to pseudo-optical images. By incorporating deep features from a multiscale feature learning module (MFLM) as semantic constraints, the translated images preserve structural details and reduce modality discrepancies. For feature matching, a channel attention-based hierarchical aggregation module (CA-HAM) is designed to effectively fuse multi-level features. Combined with a joint detection-description strategy, the network enables accurate keypoint detection and descriptor extraction. Experiments on optical-SAR datasets show that the proposed method achieves an average registration error of 2.26 pixels, outperforming state-of-the-art (SOTA) methods and enabling accurate registration between optical and SAR images.