Optimized sequential classification models for mild cognitive impairment screening based on handwriting and speech data

BackgroundHandwriting and speech are served as reliable signatures for detecting cognitive decline, playing a pivotal role in the early diagnosing Alzheimer's disease (AD) and mild cognitive impairment (MCI). However, current unimodal approaches for diagnosing AD and MCI have demonstrated constraints in classification accuracy, potentially overlooking the synergistic value of combining handwriting and speech data.ObjectivePresenting an innovative multi-modal screening classification model, that harnesses handwriting and speech analysis to enhance MCI detection, aiming to overcome the constraints of single-modality approaches by integrating data from both modalities, thereby improving diagnostic accuracy.MethodsProposing a multimodal classification model based on gated recurrent unit (GRU) and attention mechanism, treating handwriting and speech data as sequence inputs. The model was constructed and tested on a dataset of 41 participants, including 20 MCI patients and 21 cognitively normal (CN) individuals. To mitigate the risk of overfitting due to the small sample size, we employed a 10-fold cross-validation strategy to ensure the robustness of the results.ResultsOur multimodal classification model achieved an accuracy of 95.2% for MCI versus CN individuals, which shows a significant improvement compared to the results of single-modality. This result indicates the effectiveness of the cross-fusion model in enhancing classification performance, offering a promising approach for the early diagnosis of neurodegenerative diseases.ConclusionsThe proposed GRU_CA effectively improves early MCI detection by fusing handwriting and speech data, outperforming a single modality. It shows strong potential for deployment in primary healthcare settings and establishes a foundation for future research on more complex diagnostic tasks, including CN, MCI, and AD classification, as well as longitudinal studies.