Mindful Assistant: Personalized Assistance in Daily Living Activities for Individuals with Paralysis via Multimodal Cognitive Fatigue Assessment.

The increasing integration of assistive robots in aiding individuals with paralysis underscores the need for systems that adapt to users’ varying cognitive states. This paper introduces a novel framework for cognitive fatigue assessment and adaptive robotic assistance in Activities of Daily Living (ADLs), leveraging multi-sensory data collection and machine learning techniques. The proposed system employs a three-tiered adaptive framework that dynamically adjusts robotic assistance based on real-time cognitive fatigue levels, transitioning between Fully Controlled Mode (FCM), Semi-Autonomous Mode (SAM), and Fully Autonomous Mode (FAM). To evaluate its effectiveness, we conducted experiments with 11 participants, implementing a comprehensive data collection protocol that included EEG, ECG, and EDA signals, along with behavioral markers. Cognitive fatigue was induced using the N-back task protocol and quantified via the Visual Analog Scale of Fatigue (VAS-F). An LSTM-based classifier achieved 85.7% prediction accuracy and 87% recall in detecting cognitive fatigue. Additionally, a Large Language Model (LLM) was integrated for natural language processing, achieving high accuracy in command interpretation across various categories. User studies showed an overall favorable experience (M = 4.39, SD = 0.70), with particularly high ratings for Task Efficiency (M = 4.63, SD = 0.53) and Usability (M = 4.43, SD = 0.69). Overall, the paper demonstrates promising preliminary performance in action-object identification and context-aware assistance for individuals with paralysis, and highlights future directions for improving command interpretation and contextual understanding.