Enhanced Alzheimer’s Diagnosis with a Lightweight Transformer: A Multimodal Fusion of Sagittal MRI and Clinical Data

Alzheimer’s disease (AD) is a major global health challenge, and early diagnosis is crucial for effective treatment. However, traditional diagnostic methods struggle to extract meaningful patterns from medical data. This study proposes a deep learning approach that integrates brain magnetic resonance imaging and structured clinical data to improve classification accuracy. We fine-tune MobileViT, a hybrid model combining convolutional neural networks (CNNs) and transformers, to analyze sagittal MRI slices, which capture longitudinal brain structures but are underutilized due to lower classification performance and fewer labeled datasets. Additionally, we integrate TabNet to process structured clinical data, including patient age, Socioeconomic Status (SES), Mini-Mental State Examination (MMSE), Clinical Dementia Rating (CDR), and volumetric brain measures. By combining MRI and clinical data through multimodal learning, our approach achieves a peak test accuracy of 99% on the OASIS-1 (416 participants) and OASIS-2 (150 participants) datasets, using a 70/15/15 train-validationtest split. This outperforms unimodal methods, where MRI-only and clinical-data-only models achieve 94% and 97% accuracy, respectively.