Infrared and visible image fusion via cross-modal differential and dual-axis attention network

Abstract Infrared and visible image fusion aims to integrate complementary multi-modal data to generate a high-quality image that preserves salient information from both modalities. Current techniques encounter challenges in effectively discriminating features unique to each image type. Furthermore, existing attention mechanisms exhibit limitations in simultaneously integrating features from spatial and channel dimensions. To address these challenges, we propose a dual-branch fusion framework, designated as the cross-modal differential and dual-axis attention (DA) network. This framework comprises two primary branches, the global enhancement (GE) branch and the multi-scale enhancement (ME) branch. The GE branch utilizes global contextual information to augment cross-modal features, whereas the ME branch processes information across multiple spatial scales. To enhance the model’s capability to focus on critical spatial and channel dimensions, a DA module is meticulously designed. In addition, a cross-modal differential attention module is introduced to capture differential features and distinctive information between the two modalities. Comprehensive experiments conducted on public datasets demonstrate that the proposed framework achieves superior performance. Both qualitative and quantitative evaluations confirm that it surpasses existing state-of-the-art methods. The code is available at https://github.com/lishuohui123/CDDANet .