FDFuse: Infrared and Visible Image Fusion Based on Feature Decomposition

Infrared and visible image fusion (IVIF) aims to integrate the information from source images into a single image, achieving a comprehensive representation of the scene. Existing methods typically focus on either the correlation or complementarity (uncorrelated) between different modalities; however, both aspects are equally important for image fusion. Hence, in this article, we propose a novel IVIF method based on feature decomposition to enhance both the correlation and complementarity of information, which is termed as FDFuse. First, FDFuse employs dual-branch encoders with distinct structures to decompose the features of source images into modality-shared and modality-specific components. We then introduce differently designed cross-attention mechanisms to enhance the correlation between modality-shared features and complementarity between modality-specific features. We further propose different fusion strategies for these two types of features to preserve important information in the fused results. Finally, the fused features are concatenated and fed into an image decoder to generate the final fused image. Extensive experiments conducted on public datasets demonstrate that FDFuse outperforms state-of-the-art (SOTA) IVIF methods, exhibiting superior performance across multiple metrics. We also show that FDFuse can improve the performance in downstream object detection tasks under the same experimental settings. The code is available at: <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/cheng411523/FDFuse</uri>