An Infrared and Visible Image Fusion Method Based on Semantic-Sensitive Mask Selection and Bidirectional-Collaboration Region Fusion

Mask is considered as an important prior for fusion, which could selectively enhance specific regions to generate ideal fused images. However, masks used in the existing methods exhibit limitations in the precise representation of targets, and more importantly, these masks are generated from a single modality, which restricts the effective integration of multi-modal information. To address this issue, we propose a competitive mask-guidance fusion method for infrared and visible images. A multi-modal semantic-sensitive mask selection network is proposed to generate complementary-mask maps, which organically integrate advantageous target regions of different modalities by competitively comparing the qualities of masks. In this network, a pseudosiamese architecture is designed to obtain respective target masks, and specifically, a spatial-aligned-based feature aggregation module is devised to produce high-quality pseudo-labels which are served as references for the generation of the complementary-mask maps. Furthermore, we propose a bidirectional-collaboration region fusion strategy, which enhances the expression of advantageous target regions from each modality inforeground while suppressing the contribution of corresponding regions from the other modality in background. Compared to methods on public datasets, the results show that our method significantly enhances the description of semantic-sensitive targets in fused images, including the saliency and the integrity of structural information. Code are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/xbsj-cool/MSCRFusion</uri>.