An Infrared-Visible Image Fusion Network with Multi-Scale Convolutional Attention and Transformer

Infrared-visible image fusion (IVIF) obtains an information-rich fused image by fusing the intensity information in infrared images and the texture information in visible light images. However, existing methods usually have difficulty in effectively balancing global and edge feature extraction while maintaining cross-modal consistency, resulting in unsatisfactory fusion quality. In this paper, a cross-modal multi-scale dual-branch feature fusion network named MTFuse is proposed to address the key challenges of IVIF, including global and edge feature extraction, cross-modal information fusion, and highlighting important features. The features of the proposed method include a Transformer-CNN framework for integrated feature extraction, a multi-scale convolutional attention fusion block (MCAFB) for improved detail preservation, and a novel loss function inspired by focal loss for highlighting key areas in the fused image. Experimental results on benchmark datasets show that our method performs well on various metrics and significantly improves the fusion quality and effectiveness.