Engagement Patterns with an AI Health Coach for Systemic Sclerosis Self‐Management: A Mixed Methods Study

OBJECTIVE: To evaluate utility of an artificial intelligence (AI) health coach for systemic sclerosis (SSc) self-management and identify patterns associated with participant engagement. METHODS: We conducted a mixed-methods study in which an AI health coach, powered by a large language model (LLM), was used to support self-management for SSc. Twenty individuals with SSc interacted with the AI health coach over 4-weeks. Quantitative usage metrics (number of conversations, user messages, and chat duration) were used to classify participants into high- and low engagement groups. Qualitative inductive content analysis of chat transcripts was used to identify the purpose of interactions. Quantitative and qualitative data were integrated using a joint display approach. RESULTS: Twenty participants (90% female; mean age 55 years) used the AI health coach for goals and strategies, information seeking, companionship, and disease monitoring. Participants with high engagement (N=8) had more coded interactions related to goals and strategies (23.6 vs 5.1), information seeking (10.8 vs 3.9), and companionship (7.9 vs 1.0) compared to those with low engagement (N=12). More participants in high engagement group used the AI health coach for companionship compared to the low engagement group (100% vs 33%). Exploratory analyses suggested greater improvements in fatigue (mean change -5.04 [95% CI -9.34, -0.73]) among high engagement participants, with no statistically significant between-group differences. CONCLUSION: Engagement with an AI health coach may not be fully captured by quantitative usage metrics alone. Engagement in our study was characterized by more frequent, action-oriented, and companionship-oriented interactions.