Multimodal image fusion based on semantic guidance and multitask synchronous optimization

Multimodal image fusion, specifically Infrared and Visible Image Fusion (IVIF), merges visible light textures with infrared thermal data to enhance perception for applications like object detection and night vision. Existing methods often prioritize visual quality over utility in downstream machine vision tasks. We propose a novel framework using semantic guidance, contrastive learning, and a pre-trained large vision model (LVM) to improve modality-specific feature perception. A meta-learning approach jointly optimizes image fusion and downstream tasks, supported by a loss-generating network. Our method outperforms three state-of-the-art approaches in both fusion and downstream task performance, which proves its effectiveness.