FSGFuse: Feature synergy-guided multi-modality image fusion

The purpose of infrared and visible image fusion (IVIF) is to integrate the complementary properties of the two modalities to obtain a fused image with a more comprehensive information representation. However, existing multi-modal image fusion (MMIF) methods mainly rely on auto-encoders to extract source features and achieve feature complementarity through fusion mechanisms. These methods fail to fully utilize the potential of the encoder in feature decoupling and information integration , resulting in difficulties in effectively capturing and fusing cross-modal features, which in turn introduces redundant information. To tackle the challenge, this study proposes a Feature Synergy-Guided Fusion (FSGFuse) network, which achieves IVIF by synergizing complementary features to preserve texture details and acquire thermal target information . In the pre-training stage, the feature synergism loss is utilized to guide the double-branch encoder to perceive the consistency and difference of cross-modal features, which in turn motivates the model to capture the more discriminatory feature representations in the source features. In the fusion stage, a Feature Complementary Fusion (FCF) module is designed to guide feature fusion . This module not only integrates complementary features, but also captures long-range contextual relationships between features, which facilitates cross-modal interaction while enriching the semantic information. Experiments on publicly available benchmark datasets show that the fusion performance of FSGFuse significantly outperforms existing state-of-the-art methods and positively affects the effectiveness of the downstream object detection task, with a mean average precision (mAP) improvement of 18.57%. This result fully demonstrates its practical application value.