A Method Noise Based-Multi-Modal Medical Image Fusion Technique Using Non-Subsampled Shearlet Transform and Convolutional Neural Network

This paper presents a novel multimodal medical image fusion framework (M3IF-NSST-CNN) combining the Non-Subsampled Shearlet Transform (NSST) and deep convolutional neural network (CNN) methods to enhance diagnostic accuracy by effectively integrating Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Initially, both CT and MRI images undergo decomposition via NSST, producing distinct low-pass (LPSB) and high-pass subbands (HPSB). The LPSB fusion leverages a correlation-based approach, ensuring preservation of structural integrity and maximization of complementary information. Concurrently, the HPSB are merged through a Universal Image Quality Index (UIQI)-based fusion rule, prioritizing edge and detail retention critical for precise medical diagnosis. Subsequently, an inverse NSST transforms these fused sub-bands, generating a preliminary fused image. This image is then further refined through a CNN-based noise reduction module comprising convolutional layers, batch normalization, and Rectified Linear Unit (ReLU) activation functions. The deep CNN architecture progressively refines the fused output, significantly reducing artifacts and noise, thus improving the overall clarity and interpretability of the resultant image. This approach addresses common limitations in image fusion, such as loss of edge details, distortion, and the inability to preserve critical information. The hybrid M3IF-NSST-CNN method improves both the visual quality and parametric accuracy of the fusion, outperforming comparable techniques. By preserving more detail and edge information, this method offers significant advancements in multi-modal medical image fusion for enhanced diagnostic insights.