Design of a Distributed Fusion Network Based on Knowledge Distillation

Given the outstanding performance of knowledge distillation networks in optimizing traditional deep neural networks, this paper proposes a distributed fusion network based on knowledge distillation. It leverages knowledge distillation techniques to partition the different branches of the distributed fusion network into teacher and student networks. Specifically, the branch modules responsible for extracting feature information from source images are designated as the teacher network, while those extracting feature information from fused images are designated as the student network. Additionally, the designed distillation loss facilitates the transfer of feature information from the teacher network to the student network, enabling the learning of feature information from source images. Extensive experimental results on public datasets demonstrate the model's promising performance in enhancing fusion quality.