Comparing The Accuracy Of Body Composition Assessment Information Produced By ChatGpt 3.5 And Claude2 AI Platforms

Artificial intelligence (AI) has become increasingly prominent in our culture. However, the impact of AI on body composition assessment (BCA) is unknown. PURPOSE: To compare the accuracy of the body composition assessment (BCA) information produced by two common AI platforms, ChatGPT 3.5 and Claude2. METHODS: Thirty-six questions were developed focusing on a variety of common BCA methods and were entered into either ChatGPT 3.5 or Claude 2. The answers were evaluated by a BCA expert for accuracy, comprehensiveness, and depth using a 5-point scale. The expert was blinded to the AI platform and evaluated the answers until each question had two agreed scores. The average of the two closest trials for each question was used for the analysis. The Wilcoxon signed-rank test was used to determine the differences for each of the 36 questions between the two AI platforms. RESULTS: In 57% of the graded answers, two trials were required to reach agreement, three trials were required for 35%, and four trials were required for 8%. Exact agreement was ultimately found in 98.6% of the cases. Claude 2 showed higher mean ranks (M = 4.21, SD = 0.64) than ChatGpt 3.5 (M = 4.03, SD = 0.64). This difference was statistically significant (Z = 2.07, p = .038). Additionally, the median score for Claude 2 (Mdn = 4.5) was higher than for ChatGpt 3.5 (Mdn = 4.0). CONCLUSIONS: The results indicate that Claude 2 scored significantly higher across the set of 37 matching questions than ChatGpt 3.5. Given an understanding of the subject matter differences between the platforms, the significant Wilcoxon signed-rank test suggests that Claude 2 has a higher level of competency in BCA-related questions. Further standardized testing may confirm the gaps in the abilities of these two AI platforms.