3D Multi-Modal Feature Interaction Fusion Network for Early Alzheimer's Disease Diagnosis

Positron Emission Tomography (PET) and Structural Magnetic Resonance Imaging (sMRI) are widely used for early Alzheimer's disease (AD) diagnosis, providing anatomical and metabolic insights. However, current fusion methods, like simple addition or concatenation, fail to leverage their complementary information fully. To address it, we proposed a 3D multi-modal feature interaction fusion network for AD diagnosis. Specifically, we leverage a robust 3D ResNet backbone network as the architecture for feature extraction. To enable the model to dynamically focus on the areas of interest, we added a local attention module at each layer of the network. We further introduced a collaborative attention module to facilitate the interaction and fusion of features between PET and sMRI modalities, thereby enhancing the overall capability of the model to utilize complementary information from both modalities. Finally, an enhanced feature fusion module based on Transformer is integrated into the network to further strengthen the representation of the fused features. The experiment shows that our model performs better than other representative models and validates the effectiveness of the proposed method on the ADNI dataset.