The Performance Of Artificial Intelligence For Answering Body Composition Questions From Researchers Versus Laypersons

Artificial intelligence (AI) is being used more in both science and the public; however, its impact on the field of body composition assessment (BCA) has not been studied. PURPOSE: To compare the accuracy of BCA information generated by AI in response to questions typically asked by researchers (RES) compared to questions asked by laypersons (LAY) using two common platforms, ChatGPT 3.5 and Claude 2. We hypothesized that AI would be more accurate for simpler BCA questions than for those that were more technical. METHODS: Questions were developed focusing on the most common BC methods and were entered into the AI platforms. The questions were either more technical (likely what a RES would ask) or more simple (likely what a LAY might ask). Each answer was evaluated for accuracy on a 5-point scale by a BC expert who was blinded to the platform. Questions were evaluated repeatedly until there were two grading trials that agreed within 0.5 points. The averages of these duplicate scores were compared between the RES and LAY questions using the Mann-Whitney U test. Chi-square tests were used to test for differences between RES and LAY in the percentage of answers ≥4 out of 5. RESULTS: For grading the answers, 57% required only two trials to reach agreement, 35% required three trials, and 8% required four trials. Nearly every answer (98.6%) agreed exactly. For the Chat GPT, the score for RES (M = 4.1, SD = 0.5, Median = 4, Range 3.25 to 5) was not different from the score for LAY (M = 3.8, SD = 1.0, Median = 4. Range 2 to 5)(p>0.05). For Claude, the score for RES (M = 4.2, SD = 0.6, Median = 4.5, Range 3 to 5) was also not different from the score for LAY (M = 4.1, SD = 0.8, Median = 4.5, Range 2.5 to 5)(p>0.05). The percentages of scores ≥4 for Chat GPT were 67% (LAY) and 78% (RES)(p>0.05), and the percentages of scores ≥4 for Claude were 67% (LAY) and 81% (RES)(p>0.05). CONCLUSIONS: Out of 100%, grades on AI’s answers ranged from 76% to 84%. Regardless of which AI platform used, scores were the same on more technical RES questions as on LAY questions for BCA. AI gives moderately accurate answers to BCA questions with no difference in accuracy between technical questions for RES and simpler questions for LAY. AI was used to answer Body Composition questions. Determining its accuracy was the purpose of the study.