MCCAR: multi-branch convolutional and confidence-aware registration for infrared-visible images

Infrared-visible image registration is a fundamental preprocessing step for multimodal image fusion, object detection, and complex scene understanding. However, due to the heterogeneous imaging mechanisms of infrared and visible sensors, the two modalities exhibit pronounced non-linear radiometric differences, weakly textured regions, and local occlusions, which make cross-modal geometric correspondence estimation challenging. To address the susceptibility of existing methods to mismatches in unreliable regions and their limited ability to characterize fine-grained geometry in low-texture areas, this paper proposes a confidence-aware infrared-visible image registration network with Multiple Local Detail Convolution, termed MCCAR. Specifically, a CNN-Transformer encoder is first employed to extract modality-shared structural representations and modality-specific detail features. Then, a multiple local detail convolution module is designed to enhance cross-modal geometric cues from multi-scale structures, omnidirectional edges, and Laplacian high-frequency residuals. Furthermore, a pixel-wise confidence estimation branch is introduced to adaptively assess regional matching reliability under occlusion, thermal contrast variation, and texture degradation, with the resulting confidence map serving as a soft mask in the loss function to suppress unreliable correspondences during optimization. Finally, hard negative contrastive learning is adopted to improve the discriminative capability of the network for subtle geometric deviations. Experiments on the RoadScene dataset demonstrate that MCCAR achieves superior registration performance in terms of mutual information, correlation coefficient, and Mean Square Error, while further improving the quality of registration-guided multimodal image fusion.

Authors

Institutions

Publication Details

Journal
Frontiers in Computer Science
Published
2026-11-18
DOI
https://doi.org/10.3389/fcomp.2026.1933491
Primary Topic
Advanced Image Fusion Techniques
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

MCCAR: multi-branch convolutional and confidence-aware registration for infrared-visible images

Shaojian Qiu, Shaoliang Fang, Lu Lu, Wenhui Chen et al.
3 links
Frontiers in Computer Science
Advanced Image Fusion Techniques
article

MCCAR: multi-branch convolutional and confidence-aware registration for infrared-visible images

Shaojian Qiu, Shaoliang Fang, Lu Lu, Wenhui Chen, Jinsen Sun
article en

Abstract

Infrared-visible image registration is a fundamental preprocessing step for multimodal image fusion, object detection, and complex scene understanding. However, due to the heterogeneous imaging mechanisms of infrared and visible sensors, the two modalities exhibit pronounced non-linear radiometric differences, weakly textured regions, and local occlusions, which make cross-modal geometric correspondence estimation challenging. To address the susceptibility of existing methods to mismatches in unreliable regions and their limited ability to characterize fine-grained geometry in low-texture areas, this paper proposes a confidence-aware infrared-visible image registration network with Multiple Local Detail Convolution, termed MCCAR. Specifically, a CNN-Transformer encoder is first employed to extract modality-shared structural representations and modality-specific detail features. Then, a multiple local detail convolution module is designed to enhance cross-modal geometric cues from multi-scale structures, omnidirectional edges, and Laplacian high-frequency residuals. Furthermore, a pixel-wise confidence estimation branch is introduced to adaptively assess regional matching reliability under occlusion, thermal contrast variation, and texture degradation, with the resulting confidence map serving as a soft mask in the loss function to suppress unreliable correspondences during optimization. Finally, hard negative contrastive learning is adopted to improve the discriminative capability of the network for subtle geometric deviations. Experiments on the RoadScene dataset demonstrate that MCCAR achieves superior registration performance in terms of mutual information, correlation coefficient, and Mean Square Error, while further improving the quality of registration-guided multimodal image fusion.

Frontiers in Computer ScienceVol. 8
South China Agricultural University (CN), South China University of Technology (CN)
Special Project for Research and Development in Key areas of Guangdong Province
Advanced Image Fusion Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon