DAGANFuse: Infrared and Visible Image Fusion Based on Differential Features Attention Generative Adversarial Networks

The purpose of multi-modal visual information fusion is to integrate the data of multi-sensors to generate an image with higher quality, more information, and greater clarity so that it contains more complementary information and fewer redundant features. Infrared sensors detect thermal radiation emitted by objects, which is related to their temperature, whereas visible light sensors generate images by capturing the light that interacts with objects, including reflection, diffusion, and transmission. However, due to the different principles of infrared and visible light sensors, there is a large similarity difference between the generated infrared and visible images, which makes it difficult to extract complementary information. Existing methods generally use simple splicing or addition methods to fuse features at the fusion layer without considering the intrinsic features of different modal images and the interaction of features between different scales. Moreover, only correlation is considered. On the contrary, the image fusion task needs to pay more attention to their complementarity. For this reason, we introduce a cross-scale differential features attention generative adversarial fusion network, namely DAGANFuse. In the generator, we designed a cross-modal differential features attention module to fuse the intrinsic content of different modal images. We proposed a parallel path calculation of differential features and fusion features for attention weights and performed parallel spatial and channel attention weight calculations on the two paths. In the discriminator, a dual discriminator was used to maintain the information balance between different modalities and avoid common problems such as information blurring and loss of texture details. Experimental results show that our DAGANFuse has state-of-the-art (SOTA) performance and is superior to existing methods in terms of fusion performance.