FCSD-Net: A fourier convolution and state-space duality network for multi-modal medical image fusion

Abstract In the field of multi-modal medical image fusion (MMIF), combining images from different sources generates a more comprehensive, information-rich fused image. The aim is to improve the quality of clinical diagnostic assistance by providing a more reliable image. To address the aforementioned challenges, we proposed a fourier convolution and state-space duality network (FCSD-Net) for multi-modal medical image fusion. Initially, the image is transformed from the spatial domain to the frequency domain using fourier convolution. This simplifies convolution calculations and effectively extracts image texture and structure, and spatial domain and frequency domain features. Then, we suggested the hidden state interweaver-based state space duality (HSI-SSD), which maps input features onto the hidden state space to reveal task-relevant latent state features. This HSI-SSD extracts deep semantic information while reducing computational complexity. Subsequently, we put forward a dual attention feature discrimination enhancement module (DFEM) that combines dynamic and spatial attention weights. The DFEM improves the ability to discriminate between features and highlights key regions. Ultimately, we advanced the dual-domain fusion mechanism (DDFM), which enhances the unique functional and textural features of both the spatial and frequency domains, while also achieving cross-modal deep fusion. The experimental results on the PET-MRI dataset demonstrate that FCSD-Net achieves AG, SF, SD, Qabf, and $$\varvec{SSIM_{A,B/F}}$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mi>S</mml:mi> <mml:mi>S</mml:mi> <mml:mi>I</mml:mi> <mml:msub> <mml:mi>M</mml:mi> <mml:mrow> <mml:mi>A</mml:mi> <mml:mo>,</mml:mo> <mml:mi>B</mml:mi> <mml:mo>/</mml:mo> <mml:mi>F</mml:mi> </mml:mrow> </mml:msub> </mml:mrow> </mml:math> values of 11.4450, 35.1403, 88.5587, 0.7166, and 1.3386 respectively. FCSD-Net can produce fused images with more comprehensive detail and richer content. This providing more reliable and precise support for clinical decision-making.