HiFusion: An Unsupervised Infrared and Visible Image Fusion Framework With a Hierarchical Loss Function

Infrared and visible image fusion (IVIF) aims to integrate useful information from different source images into a single fused image. Although the performance of deep learning (DL)-based fusion methods has significantly improved in recent years, most of these methods empirically design fusion networks to enhance performance while neglecting constraints on the fusion network, which limits its capability, particularly in terms of generalization across different fusion scenarios and adaptability to downstream tasks. In this article, we propose a novel unsupervised fusion framework with a hierarchical loss function for IVIF, named HiFusion. This framework consists of an end-to-end fusion network to directly generate fused images and a constraint network consisting of two discriminators and a pretrained autoencoder to constrain the information between the fused and source images at the modal, pixel, and detail levels. The losses calculated at each level are coherently integrated into the hierarchical loss function, thereby achieving comprehensive constraints on the fusion network. Extensive experiments on several datasets demonstrate the superiority of our proposed method over other state-of-the-art fusion methods, especially in terms of medical image fusion and downstream tasks, indicating the excellent adaptability and extensibility of our method. Our codes will be publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/PCwenyue</uri>.