Weakly Supervised Cross Mixer for Infrared and Visible Image Fusion

Remote sensing infrared and visible image fusion aims to integrate information from multiple source images to enhance visual representation and support high-level visual tasks. However, due to the lack of ground truth supervision, most existing methods depend on predefined fusion-specific loss functions. Such manual definitions of cross-modal feature representations are often incomplete, potentially leading to information loss and reduced fusion performance. Complete information retention in fusion results can significantly benefit high-level tasks. Motivated by this, we propose a novel approach that utilizes weakly-supervised segmentation guidance for comprehensive information representation ability, eliminating the need for fusion-specific losses. Firstly, a self-supervised reconstruction model is proposed to obtain a decoder with robust cross-modal feature representation, which can adaptively reconstruct images based on cross-modal features. Then we design a weakly-supervised feature mining module to capture fused features with complete cross-modal information representation under the guidance of the segmentation task. The finial fusion result is adaptively reconstructed from the fused features by the cross-modal adaptive decoder without using any constraints. Extensive experiments demonstrate that the proposed method effectively mitigates information loss and achieves superior performance in both fusion and segmentation tasks compared to the state-of-the-art methods. Model and code are available at https://github.com/wangwenbo26/WSCM.