Classification and diagnosis model for Alzheimer’s disease based on multimodal data fusion

Alzheimer disease (AD) is the most commonly occurring neurodegenerative disease. However, current diagnostics for AD primarily rely on invasive tests, which limit the application of diagnostic procedures in early screening. Speech, as a noninvasive biomarker, is closely associated with AD but has not been fully leveraged as a diagnostic tool. This study develops a novel early AD diagnosis method that uses primitive speech and explores its potential application in community screening. Moreover, the study proposes an innovative multimodal method for speech feature fusion that combines acoustic and semantic information to differentiate patients with AD from normal controls. This method uses the ImageBind audio encoder to extract acoustic features and the Embeddings from Language Model to extract semantic features, thereby effectively integrating the features by mid-level fusion. The training set comprises 166 speech recordings, which comprise 87 samples from individuals with AD and 79 samples from healthy control subjects. The ratio of training set to test set is 7:3. Evaluation of the Alzheimer dementia recognition through spontaneous speech only dataset showed that the proposed model achieved a classification accuracy of 0.903 and a recall rate of 1, and it considerably outperformed existing baseline models, thereby confirming the effectiveness of the proposed approach to AD diagnosis. This study applies the multimodal fusion of speech features to an early AD diagnostic procedure and achieves excellent performance. The findings of this study not only provide a new approach to noninvasive AD screening but also open new pathways to the early diagnosis of other neurodegenerative diseases.