Transformer-based infrared and visible image fusion with language-driven loss

Infrared and visible image fusion aims to generate a fused image that highlights the target and encompasses detailed textural information. Deep learning-based image fusion algorithm enable enhanced multimodal sensing and decision-making across diverse environments. However, existing algorithms often focus only on visual quality while neglecting the image's semantic content. To address these issues, we propose an image fusion method that leverages the global feature extraction capabilities of the transformer model and optimises the fusion process through a loss function guided by the contrastive language-image pre-training (CLIP). Firstly, we develop a feature-guided transformer (FGT) module to extract and interact with local and global information in images. Subsequently, a feature dynamic fusion (FDF) module is designed to fuse the image adaptively from two different modalities. Comprehensive experiments conducted on three public datasets demonstrate that the proposed method outperforms state-of-the-art (SOTA) fusion methods subjectively and objectively.