MDMF-Net: Multi-Dimensional Integrated Multimodal Feature Fusion Alzheimer's Disease Prediction Network*

Different modalities of neuroimaging data can provide complementary lesion information for the diagnosis of Alzheimer's disease. However, existing methods face trade-offs in obtaining three-dimensional lesion features and suppressing redundant 3D features. It is difficult to simultaneously preserve the ability of 2D networks to reduce redundant features and lower training costs while extracting features of the 3D lesion regions with spatial structure. To address this challenge, this paper proposes a multi-dimensional integrated multimodal feature fusion AD prediction network (MDMF-Net) to obtain low-redundancy lesion information with 3D spatial structural features. First, a multi-dimensional joint feature extraction module is proposed, where a 2D network is used to acquire 2D lesion features and generate a saliency feature map. Based on the saliency feature map, key lesion blocks are segmented from the 3D brain images to extract 3D lesion features. Secondly, a multimodal multi-view perception feature fusion module is designed to reduce heterogeneity of features from different modalities and dimensions through various attention mechanisms and a dual-stage fusion strategy. Finally, a saliency feature loss is introduced to enhance the saliency response of the key lesion regions while suppressing the response from irrelevant regions. Experimental results on the ADNI database show that our model achieves a classification accuracy of 89.12 %, outperforming several competitive methods in terms of prediction performance.