Comparing Artificial Intelligence Platforms' Accuracy In Weight Loss And Exercise Science Recommendations

Artificial intelligence (AI) has become increasingly prominent in providing health and fitness advice. However, the accuracy of AI-generated recommendations for weight loss remains unexplored. PURPOSE: To compare the accuracy and completeness of weight loss information produced by two AI platforms, ChatGpt 4.0 (CHAT), and Claude 3.5 Sonnet (CLAUDE). METHODS: Fifteen questions spanning four categories were entered into both AI platforms. A Ph.D. expert in weight management blinded to the platforms evaluated responses using two criteria: accuracy/evidence-based content (0-4 points) and completeness/depth (0-4 points). Questions were grouped: fundamental principles (5 at 20 points), dietary interventions (3 at 12 points), exercise activity (5 at 20 points), and medical interventions (2 at 8 points). Statistical analyses included Wilcoxon signed-rank tests, Cohen's d effect sizes, and Spearman correlations between accuracy and completeness scores. RESULTS: In fundamental principles, CLAUDE outperformed CHAT in both accuracy (13/20 vs 11/20, p = 0.042) and completeness (11/20 vs 9/20, p = 0.039). For dietary interventions, CLAUDE showed higher scores in accuracy (10/12 vs 9/12, p = 0.046) and completeness (7/12 vs 6/12, p = 0.048). Exercise activity showed differences in accuracy (CLAUDE 3.5 11/20 vs CHAT, p = 0.044) but not completeness (both 7/20, p = 0.956). Medical interventions showed equal accuracy (both 6/8, p = 1.000) but different completeness scores (CLAUDE 6/8 vs CHAT 4/8, p = 0.041). Effect sizes were moderate for both accuracy (d = 0.68) and completeness (d = 0.72). Accuracy and completeness scores showed strong correlation in CLAUDE (r = 0.79, p = 0.002) but moderate correlation in CHAT (r = 0.54, p = 0.031). CONCLUSIONS: CLAUDE showed superior performance across most categories, with moderate effect sizes supporting meaningful differences between platforms. The strong correlation between accuracy and completeness in CLAUDE suggests more consistent performance compared to CHAT. While both platforms showed adequate capability in medical and dietary domains, the exercise activity scores emphasize the need for continued development in exercise prescription capabilities. Therefore, when using AI platforms for weight loss information, expert oversight is of utmost importance.