Multimodal Medical Image Feature Representation and Fusion for AD Early Diagnosis

Alzheimer's disease (AD) is progressive and gets worse with time. Early detection is an effective treatment for the disease, which can timely implement effective interventions. The multi-modal medical image contains more comprehensive and abundant information about the disease, which can reflect the different aspects of AD and reduce the potential bias in single-modal diagnosis. However, an effective and reliable solution is still lacking, which can integrate multimodal neuroimaging effectively for the precision diagnosis and treatment of AD. In this work, we proposed a feature representation and fusion model of MRI and PET scans based on a multiscale convolution network and cross-attention for AD early diagnosis, named camAD. Compared with the existing models, camAD performs better with fewer pre-processing steps. Using the Alzheimer’s disease neuroimaging initiative (ADNI) datasets, we demonstrated that integrating multi-modality data outperforms single-modality models in accuracy, specificity, sensitivity, AUC, and F1 scores. Our models exhibited significantly good accuracy and generality in four classification tasks, which provided a promising way to understand the underlying mechanisms of the disorder's progression.