DPSNET: A Dual-Path Lightweight Network With Semantic-Guided Cross-Modal Feature Fusion for Multimodal Object Detection

Multimodal object detection is crucial for autonomous driving and intelligent surveillance. However, existing methods face critical challenges, such as cross-modal semantic misalignment from heterogeneous spatial structures, degraded infrared representation under low contrast, and inadequate adaptation of visible feature extraction. To fix the gap, we propose a lightweight dual-path network with semantic-guided cross-modal feature fusion for multi-modal object detection. Firstly, to mitigate cross-modal feature-level semantic misalignment, the Semantic-Guided Cross-Modal Fusion (SCMF) module based on a semantic-guided attention is designed, which constructs channel-specific spatial importance maps and enables fine-grained and semantically consistent fusion of heterogeneous features via structured attention and adaptive residual modulation. Secondly, the Adaptive Feature Enhancement Aggregation (AFEA) module is proposed to jointly capture long-range semantic context and fine-grained local structure features by combining global attention inference, local feature estimation, and spatially aware receptive field enhancement. Lastly, the Dynamic Receptive Field Attention (DRAF) module is put forward, which dynamically adjusts feature interactions based on adaptive receptive field modulation to further refine feature-specific fusion, improving robustness under diverse conditions. Experimental results on LLVIP, FLIR and KAIST datasets demonstrate that DPSNET maintains a good balance between detection accuracy and model lightweight, making it a viable solution for multimodal scenarios.

Authors

Institutions

Publication Details

Journal
IEEE Transactions on Instrumentation and Measurement
Published
2025-01-01
DOI
https://doi.org/10.1109/tim.2025.3612628
Citations
1
Primary Topic
Advanced Image and Video Retrieval Techniques
Type
article
Field-Weighted Citation Impact
1.28
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

DPSNET: A Dual-Path Lightweight Network With Semantic-Guided Cross-Modal Feature Fusion for Multimodal Object Detection

Haiyan Li, Jundong Yang, Dan Xu, Yue Geng et al.
1 citations
IEEE Transactions on Instrumentation and Measurement
Advanced Image and Video Retrieval Techniques
1.28
article

DPSNET: A Dual-Path Lightweight Network With Semantic-Guided Cross-Modal Feature Fusion for Multimodal Object Detection

Haiyan Li, Jundong Yang, Dan Xu, Yue Geng, Guanbo Wang, Yu Huang, Xiang Liu
article en
1 citations

Abstract

Multimodal object detection is crucial for autonomous driving and intelligent surveillance. However, existing methods face critical challenges, such as cross-modal semantic misalignment from heterogeneous spatial structures, degraded infrared representation under low contrast, and inadequate adaptation of visible feature extraction. To fix the gap, we propose a lightweight dual-path network with semantic-guided cross-modal feature fusion for multi-modal object detection. Firstly, to mitigate cross-modal feature-level semantic misalignment, the Semantic-Guided Cross-Modal Fusion (SCMF) module based on a semantic-guided attention is designed, which constructs channel-specific spatial importance maps and enables fine-grained and semantically consistent fusion of heterogeneous features via structured attention and adaptive residual modulation. Secondly, the Adaptive Feature Enhancement Aggregation (AFEA) module is proposed to jointly capture long-range semantic context and fine-grained local structure features by combining global attention inference, local feature estimation, and spatially aware receptive field enhancement. Lastly, the Dynamic Receptive Field Attention (DRAF) module is put forward, which dynamically adjusts feature interactions based on adaptive receptive field modulation to further refine feature-specific fusion, improving robustness under diverse conditions. Experimental results on LLVIP, FLIR and KAIST datasets demonstrate that DPSNET maintains a good balance between detection accuracy and model lightweight, making it a viable solution for multimodal scenarios.

IEEE Transactions on Instrumentation and MeasurementVol. 74
Yunnan University (CN)
Openalex Percentile: Top 15%
Advanced Image and Video Retrieval Techniques
1.28
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.