Assessing AI-based chatbots accuracy in caloric estimation: A focus on traditional Turkish foods

Objectives: To evaluate the accuracy of three widely used AI-based chatbots - ChatGPT 4.0, Microsoft Copilot, and Gemini - in estimating the caloric values of traditional Turkish foods. Methods: The accuracy of caloric information provided by the chatbots for 71 traditional Turkish foods selected from the Türkomp National Food Composition Database was assessed. Each chatbot was queried in Turkish using the standardised prompt: "What is the caloric content of [food name] per 100 grams?" Responses were recorded and, when necessary, converted to a per-100-gram basis to ensure consistency. Accuracy percentages were calculated by comparing chatbot responses to Türkomp reference values, and extreme deviations were adjusted to 0%. Mean accuracy scores and distribution across predefined accuracy intervals were analysed for each chatbot. Statistical analysis was conducted to determine the mean accuracy percentages and identify differences among the chatbots. Results: ChatGPT 4.0 achieved the highest mean accuracy (81.62%±20.6%), followed closely by Microsoft Copilot (81.23%±20.7%), while Gemini demonstrated lower accuracy (70.99%±30.2) (P<0.05). A one-way ANOVA showed a statistically significant difference in the mean accuracy percentages among the chatbots (F(2, 210)=4.39, P=0.0136). Foods such as kefir, tahini halva, and walnut baklava were estimated with over 90% accuracy by all three chatbots, suggesting strengths in their training datasets and the relatively simple or standardised nutrient composition of these foods. However, significant discrepancies in caloric estimations were observed across the chatbots, likely due to differences in algorithms and database integrations. Conclusions: The findings suggest that AI-based chatbots have the potential to serve as culturally relevant tools for dietary assessment. However, the results also emphasise the need for careful use and further development. While ChatGPT 4.0 and Microsoft Copilot performed better than Gemini, the study shows the need for improved algorithms and expanded training datasets to enhance the accuracy and reliability of chatbots in nutritional evaluation. This study contributes to the growing body of research on AI applications in dietetics and public health; however, addressing these limitations is crucial to ensure their practical utility. Optimising chatbot design for real-world use will require interdisciplinary collaboration among AI developers, nutritional scientists, and healthcare professionals.