Comparative Evaluation of Artificial Intelligence-Generated Physician-Level Educational Content in Diabetic Foot Ulcers: Quality, Guideline Concordance, and Clinical Reliability

The study aimed to evaluate the quality, clinical reliability, and guideline concordance of physician-level educational responses generated by artificial intelligence (AI) in diabetic foot ulcer management. A blinded, comparative analysis was conducted using three widely used large language models—ChatGPT, Gemini, and Claude—which were assessed across twelve standardized clinical questions covering prevention, diagnosis and classification, and treatment. All questions were developed based on an established clinical guideline (International Working Group on the Diabetic Foot, IWGDF). Responses were independently evaluated by six expert raters using predefined criteria, including clinical depth, technical accuracy, decision-making utility, overall quality, and the presence of potentially harmful or misleading information. Guideline concordance was assessed using structured checklists derived from established recommendations. Significant differences were observed among the evaluated models across all domains. Gemini consistently demonstrated higher performance in clinical depth, technical accuracy, decision-making utility, and overall quality, while also showing greater alignment with guideline-based recommendations. ChatGPT and Claude demonstrated comparatively lower scores and a higher tendency to produce responses containing potentially misleading information. Inter-rater reliability analysis indicated moderate to good agreement among evaluators. These findings suggest that, although artificial intelligence systems show considerable potential in generating clinically relevant information, their performance remains variable and may not consistently meet the standards required for high-risk clinical decision-making. Careful evaluation and validation are therefore essential before integrating such systems into routine clinical practice.