A Study on the Fusion of Infrared and Visible Images Using Edge Convolution with Transformer in a Dual-Branch Structure

Currently, most deep learning-based methods for infrared and visible image fusion focus on feature extraction from source images and the design of fusion network loss functions while neglecting the inherent differences between infrared and visible images. In this study, the author examined the characteristics of infrared and visible images, explored their shared and cross-modal features, and proposed a dual-branch fusion method for these two modalities. Specifically, the feature encoding of the fusion network was divided into two branches: a cross-modal feature extraction branch and a shared feature extraction branch. Edge convolution and transformer modules were introduced on different branches to extract cross-modal and shared information, thereby separating complementary cross-modal features and redundant shared features. A convolutional attention module was introduced in the cross-modal feature branch for feature selection and local enhancement. Finally, different loss functions were designed for the proposed dual-branch encoder–decoder network. Extensive experiments demonstrated that proposed fusion method outperformed representative methods in terms of both subjective visual quality and objective metrics.