AI-Assisted Analysis of Spontaneous Affective Facial Responses for Cognitive Status Assessment in Older Adults: Exploratory Feasibility Study

BACKGROUND As Taiwan enters a super-aged society, the prevalence of dementia continues to rise, highlighting the need for scalable, noninvasive approaches to identify behavioral alterations associated with cognitive decline. AI-based analysis of spontaneous affective facial responses has emerged as a potential digital phenotyping tool, but its feasibility for assessing cognitive performance remains underexplored. OBJECTIVE This study aimed to examine the feasibility of using AI-derived spontaneous affective facial response metrics, including affective expression congruency and onset latency, to characterize differences across cognitive performance levels in older adults with healthy cognition (HC), mild cognitive impairment (MCI), or dementia. METHODS A total of 60 participants (HC, MCI, and dementia; 20 per group) were presented with positive and negative emotion-eliciting videos and completed Mini-Mental State Examination (MMSE) recall and calculation tasks. Facial expressions were video-recorded and analyzed using a convolutional neural network–based AI system. AI-derived metrics included affective expression congruency, categorized as congruent, neutral, or incongruent according to the detected facial response, and affective expression onset latency, defined as the time from stimulus presentation to the emergence of a nonneutral facial expression detected by the AI model. Group differences were analyzed using 1-way ANOVA, Pearson chi-square or Fisher-Freeman-Halton exact tests, Bonferroni-adjusted pairwise post hoc comparisons, analysis of covariance adjusting for age and education, and exploratory receiver operating characteristic analyses. RESULTS Affectively congruent responses were generally less frequent among participants with lower cognitive performance. During the 14-second positive stimulus, congruent positive expressions were observed in 65% (13/20) of HC participants, 20% (4/20) of MCI participants, and 0% (0/20) of participants with dementia. During the 48-second negative stimulus, congruent negative expressions were observed in 60% (12/20) of HC participants compared with 5% (1/20) of both MCI and dementia participants. Onset latency differed significantly across groups for the 14-second positive stimulus (F2,57=19.63; P<.001; η2=0.41), 52-second positive stimulus (F2,57=4.25; P=.02; η2=0.13), 48-second negative stimulus (F2,57=15.61; P<.001; η2=0.35), and MMSE-based cognitive-task condition (F2,57=11.28; P<.001; η2=0.28). After adjustment for age and education, the group effect remained significant for all latency outcomes (F2,55=3.89-10.68; all P≤.03; partial η2=0.12-0.28). Exploratory receiver operating characteristic analyses showed stronger separation in several HC vs MCI and HC vs dementia comparisons, whereas MCI vs dementia comparisons showed limited separability. CONCLUSIONS AI-assisted analysis of spontaneous affective facial responses revealed differences in expression congruency and onset latency across cognitive performance levels. These preliminary findings suggest that AI-derived facial behavioral metrics may serve as noninvasive adjunctive features for observational cognitive assessment. However, the findings should be interpreted cautiously because of the exploratory design, modest sample size, and absence of internal or external validation.