Evaluation of scientific validity and appropriateness of artificial intelligence-assisted ChatGPT advices in dietary treatment of methylmalonic acidemia

Background and objectives: Chat Generative Pretrained Transformer (ChatGPT) has become an increasingly popular way for patients and healthcare professionals to seek information on health and nutrition advice. Previous studies have raised concerns about the accuracy of dietary advice provided by ChatGPT. We aimed to evaluate the potential of ChatGPT as a tool to provide dietary therapy for methylmalonic acidemia (MMA) in October 2024. Methods and study design: we compared ChatGPT's (version 4.0) dietary advices for MMA with internationally recognized guidelines and current literature, and then evaluated the chatbot's ability to generate dietary therapy in two different cases with MMA. A team of dietitians and metabolic physicians compared ChatGPT responses to disease-specific guidelines and evidence-based resources. Results: overall, ChatGPT provided specific advices: 20.0 % of responses were considered appropriate and 53.4 % were considered inappropriate. Especially in this group of patients whose only known treatment was dietary therapy, the dietary therapy recommended by ChatGPT was also inappropriate for the two cases in question. Conclusions: this study was the first to evaluate the appropriateness of artificial intelligence advices for specialized dietary management of inherited metabolic disorders. Consequently, the idea that artificial intelligence can replace human labor in the near future seems far-fetched, given the low success rate of even a diagnosis with such a clear mechanism and treatment, and individualized treatment approaches.