Virtual Experience and Interactive Training Environments with Bio-signal-based Indicators for Cognitive Decline: Results of the SmartAktiv Study

Virtual Reality (VR) is among the top emerging technologies in healthcare for older adults. The SmartAktiv project developed an innovative VR-based training environment with the goal to support early detection and cognitive activation in individuals with mild cognitive impairment (MCI). The multimodal system combines immersive VR, hand and eye tracking, tablet-based exercises, and wearable biosignal sensors. Scenario development was informed by expert workshops and user focus groups. Usability testing and a pilot study evaluated system effectiveness. VR scenarios included leisure-based (I)ADLs (instrumental activities of daily living), such as, mushroom picking. Eye-tracking data and interaction performance revealed significant correlations. Participants without cognitive impairment completed tasks faster, had shorter fixations, and showed higher engagement. Findings support the potential of SmartAktiv as a motivating, sensor-driven intervention in a gamified environment with the potential for identifying digital biomarkers in early-stage MCI.The third and final phase tested the comprehensive system in a pilot study (n=30), including participants with various degrees of mild cognitive impairment. A control group (n=30) used only the Tablet training without VR. A subgroup of the recruited participants applied neurological assessment with various psychological tests led by a clinical expert in order to enable reference diagnostic information.The expert workshop identified four scenarios for further development: beach vacation, winter outing, summer hiking, and urban experience. The pilot study (n=30, female 80%, M=76.4/SD=8.2 years of age, with screening MoCAScore M=25.0/SD=3.3 ) applied these scenarios within the VR-based environment. The assessment of hand-based virtual events and eye movement features during specific sequences of the interactions resulted in significant advantages of persons without cognitive impairment (MoCA global score > 24) within IADLs, regarding the time to finalize a payment procedure, referring to faster scene understanding with lower fixation durations during natural scene observation, as well as presenting larger pupil dilation of higher engagement within entertaining scenarios. SmartAktiv emphasizes a comprehensive multimodal activation, integrating immersive and playful cognitive training within experiential scenarios. This training approach aims to strengthen all cognitive domains and offers training for various (I)ADLs within leisure scenarios. The assessment processes demonstrate statistically significant differences between cognitively impaired as well as healthy participants that. These results demonstrate the potential of the approach to provide digital sensor-based biomarkers that could be validated in future work.