GAN-Guided Few-Shot Attention Network for Medical Images Fusion Quality Assessment

Medical image fusion (MIF) plays an important role in precision diagnostics and treatment planning management, and medical image fusion quality assessment (MIFQA) has an aggressive effect in improving MIF performance. However, obtaining medical reference images is difficult, and the significant demand for medical prior knowledge and reference images is an important challenge in the field of MIFQA. To address this issue, this paper proposes a two-stage model for MIFQA. In the first stage, we design a GAN-based Quality-aware Network called QANet. By fusing the radiologist's mean opinion score (MOS) with the source image, the model is guided to generate one reference images of each quality. Then, in the second stage, the reference images are fed into our proposed class attention siamese network (CASNet) based on class activation mapping (CAM) under few-shot learning to fully explore the information in limited reference images. It can enforce the model to focus on the key lesion area and effectively reduce the dependence of MIFQA on medical fused images. Finally, the quality score of the unlabeled fused image is predicted by calculating the distance with reference image. Experiments on home-made MIFQA dataset shows that our method can achieve results that are ahead of the state-of-the-art methods.