A multimodal hybrid framework for early and accurate detection of dementia using MRI and facial imaging

Early and accurate detection of dementia, particularly Alzheimer’s disease, is critical for timely intervention and patient care. Unimodal approaches based on MRI or facial expression images face challenges such as class imbalance and limited generalization. This work introduces a novel multimodal dementia detection framework that, for the first time, integrates MRI-derived structural features with facial expression representations as an auxiliary modality. This is a multimodal hybrid diagnostic framework integrating spatial, temporal, and generative modeling with optimization-based feature refinement. The framework combines CNNs for spatial feature extraction, LSTMs for temporal modeling, and DCGAN for data augmentation to balance underrepresented classes. Four bio-inspired optimization algorithms—PSO, ACO, ABC, and WOA—were applied for feature selection and hyperparameter tuning. Publicly available MRI(OASIS-1) and facial expression(FER2013) datasets were used for evaluation. The integration of feature-level multimodal fusion, GAN-based augmentation, and bio-inspired optimization improved classification robustness and addressed limitations of individual modalities. Despite computational demands and dataset size constraints, the framework demonstrated superior generalization and reliability. The proposed hybrid framework offers a scalable and clinically relevant AI tool for dementia detection, effectively enhancing diagnostic accuracy and robustness compared to conventional unimodal methods.