YOLO-CMFM: A Visible-SAR Multimodal Object Detection Method Based on Edge-Guided and Gated Cross-Attention Fusion

To address the challenges of cross-modal feature misalignment and ineffective information fusion caused by the inherent differences in imaging mechanisms, noise statistics, and semantic representations between visible and synthetic aperture radar (SAR) imagery, this paper proposes a multimodal remote sensing object detection method, namely YOLO-CMFM. Built upon the Ultralytics YOLOv11 framework, the proposed approach designs a Cross-Modal Fusion Module (CMFM) that systematically enhances detection accuracy and robustness from the perspectives of modality alignment, feature interaction, and adaptive fusion. Specifically, (1) a Learnable Edge-Guided Attention (LEGA) module is constructed, which leverages a learnable Gaussian saliency prior to achieve edge-oriented cross-modal alignment, effectively mitigating edge-structure mismatches across modalities; (2) a Bidirectional Cross-Attention (BCA) module is developed to enable deep semantic interaction and global contextual aggregation; (3) a Context-Guided Gating (CGG) module is designed to dynamically generate complementary weights based on multimodal source features and global contextual information, thereby achieving adaptive fusion across modalities. Extensive experiments conducted on the OGSOD 1.0 dataset demonstrate that the proposed YOLO-CMFM achieves an mAP@50 of 96.2% and an mAP@50:95 of 75.1%. While maintaining competitive performance comparable to mainstream approaches at lower IoU thresholds, the proposed method significantly outperforms existing counterparts at high IoU thresholds, highlighting its superior capability in precise object localization. Also, the experimental results on the OSPRC dataset demonstrate that the proposed method can consistently achieve stable gains under different kinds of imaging conditions, including diverse SAR polarizations, spatial resolutions, and cloud occlusion conditions. Moreover, the CMFM can be flexibly integrated into different detection frameworks, which further validates its strong generalization and transferability in multimodal remote sensing object detection tasks.

Authors

Institutions

Publication Details

Journal
Remote Sensing
Published
2025-12-31
DOI
https://doi.org/10.3390/rs18010136
Primary Topic
Advanced Neural Network Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

YOLO-CMFM: A Visible-SAR Multimodal Object Detection Method Based on Edge-Guided and Gated Cross-Attention Fusion

Lijun Zhao, Keli Shi, Zheng Zhang, Ruotian Ren et al.
Remote Sensing
Advanced Neural Network Applications
article

YOLO-CMFM: A Visible-SAR Multimodal Object Detection Method Based on Edge-Guided and Gated Cross-Attention Fusion

Lijun Zhao, Keli Shi, Zheng Zhang, Ruotian Ren, Xuyang Zhao
article en

Abstract

To address the challenges of cross-modal feature misalignment and ineffective information fusion caused by the inherent differences in imaging mechanisms, noise statistics, and semantic representations between visible and synthetic aperture radar (SAR) imagery, this paper proposes a multimodal remote sensing object detection method, namely YOLO-CMFM. Built upon the Ultralytics YOLOv11 framework, the proposed approach designs a Cross-Modal Fusion Module (CMFM) that systematically enhances detection accuracy and robustness from the perspectives of modality alignment, feature interaction, and adaptive fusion. Specifically, (1) a Learnable Edge-Guided Attention (LEGA) module is constructed, which leverages a learnable Gaussian saliency prior to achieve edge-oriented cross-modal alignment, effectively mitigating edge-structure mismatches across modalities; (2) a Bidirectional Cross-Attention (BCA) module is developed to enable deep semantic interaction and global contextual aggregation; (3) a Context-Guided Gating (CGG) module is designed to dynamically generate complementary weights based on multimodal source features and global contextual information, thereby achieving adaptive fusion across modalities. Extensive experiments conducted on the OGSOD 1.0 dataset demonstrate that the proposed YOLO-CMFM achieves an mAP@50 of 96.2% and an mAP@50:95 of 75.1%. While maintaining competitive performance comparable to mainstream approaches at lower IoU thresholds, the proposed method significantly outperforms existing counterparts at high IoU thresholds, highlighting its superior capability in precise object localization. Also, the experimental results on the OSPRC dataset demonstrate that the proposed method can consistently achieve stable gains under different kinds of imaging conditions, including diverse SAR polarizations, spatial resolutions, and cloud occlusion conditions. Moreover, the CMFM can be flexibly integrated into different detection frameworks, which further validates its strong generalization and transferability in multimodal remote sensing object detection tasks.

Remote SensingVol. 18(1)
Chinese Academy of Sciences (CN), Aerospace Information Research Institute (CN), University of Chinese Academy of Sciences (CN)
Openalex Percentile: Top 34%
Advanced Neural Network Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.