Comparative Analysis of ChatGPT-Based Artificial Intelligence for Diabetes Self-Management Support: Potential of Artificial Intelligence Stratified Implementation and Customization

Objective: This study aimed to assess the effectiveness, variability, and emotional acceptability of ChatGPT-based artificial intelligence (AI) models in supporting diabetes self-management. Both common scenarios and complex cases were developed to offer deeper insights into the potential applications of AI in diabetes care. Methods: A comparative analysis was conducted using three ChatGPT-based AI models: The first one is an independently developed Diabetes Self-Management GPTs Support System, which is created with ChatGPT’s GPTs that allow users to customize it for specific purposes, while the others are the most advanced AI models currently available for general use: GPT-4 Omni and GPT-o1 Preview. Each response from the AI system was evaluated with quantitative and qualitative metrics in four case scenarios: insulin administration, an old diabetic patient with visual impairment, a pediatric patient facing stigma, and a diabetic patient on “sick day.” Furthermore, sentiment analysis using AI was conducted to evaluate the emotional tone and patient-centered language of the responses with sentiment scores ranging from −1.0 (very negative) to +1.0 (very positive). Results: The Diabetes Self-Management GPTs Support System provided concise, empathetic, and practical guidance, excelling in sentiment scores (+0.8 to +1.0); however, there was a lack of depth in complex scenarios. GPT-4 Omni delivered the most comprehensive responses with detailed medical insights, whereas its clinical tone tended to slightly lower sentiment scores (+0.7 to +0.9). GPT-o1 Preview emphasized procedural safety with moderate detail but was less empathetic (+0.5–+0.8). Across all scenarios, GPT-4 Omni consistently provided the most detailed guidance, whereas the Diabetes Self-Management GPTs Support System demonstrated superior emotional engagement. Conclusions: This study compared three large language model-based AI models for diabetes self-management. GPT-4 Omni provided the most detailed responses, Diabetes Self-Management GPTs Support System was concise and empathetic, and GPT-o1 Preview prioritized safety but lacked depth. These findings emphasize the importance of selecting AI models based on user needs and optimizing them for effective patient support.