CP-FN: A Collaborative Perception Fusion Network for Infrared and Visible Image Fusion

Multimodal feature fusion aims to generate fusion results with prominent target features and rich detail information from data of different complementary modalities. To better address key issues in multimodal feature alignment, such as channel misalignment, scale mismatch, and insufficient robustness to large-sized images, we propose a novel Collaborative Perception Fusion Network (CP-FN). The network employs a dual-stream architecture design, using a hierarchical self-attention mechanism to extract deep hierarchical multi-scale feature representations from the base stream. Combined with dilated convolution technology to effectively expand the receptive field, the extracted multi-scale information is gradually integrated into the detailed information stream to achieve feature compensation. The collaborative perception of multi-scale features progresses layer-by-layer from coarse-grained to fine-grained, promoting more comprehensive feature representation and effective cross-modal information alignment. Subsequently, we design an innovative dynamic weight allocation fusion strategy, which adaptively adjusts the weight coefficients of each modality according to the statistical characteristics of features to ensure balanced fusion of multimodal information. Additionally, through deep optimization of the self-attention mechanism, the network’s robustness and stability in processing large-sized images are significantly improved. Extensive experiments conducted on multiple benchmark datasets show that the CP-FN significantly outperforms existing comparative methods, particularly in feature alignment accuracy and fusion quality.