Evaluating the accuracy and reliability of ChatGPT in breastfeeding counseling: A literature-based assessment

In recent years, artificial intelligence (AI) tools such as OpenAI’s Generative Pre-trained Transformer (ChatGPT) have enhanced healthcare information access by enabling rapid retrieval of medical knowledge and guidelines. This study aims to evaluate the consistency, accuracy, and clinical relevance of ChatGPT’s responses on breastfeeding counseling by comparing them with current literature. The study was designed as a qualitative comparative analysis grounded in a review of current literature. Between October 21 and 23, 2024, one breastfeeding-related question per day was submitted to ChatGPT (free version, GPT-3) via the Google search engine. The questions were selected to reflect the most commonly addressed topics in breastfeeding counseling, as identified through the literature. ChatGPT’s responses were directly incorporated into the manuscript and analyzed in comparison with current literature to assess their accuracy, clarity, and sufficiency for use in patient education by healthcare professionals and laypersons. Additionally, a 5-point scoring system was applied to evaluate each response based on five content quality dimensions, such as consistency with literature and the timeliness of evidence. The data sample was limited to responses generated based on ChatGPT’s knowledge cutoff of November 30, 2022, to ensure that the content reflected a period unaffected by post-deployment user interactions. The analysis revealed that ChatGPT’s responses were generally consistent with existing guidelines and literature. However, a notable limitation was found in the response regarding breast milk storage, which did not reflect an update released on March 26, 2024. This finding illustrates ChatGPT’s limitation in reflecting the most recent evidence. While ChatGPT offers accessible and mostly accurate health information, its inability to incorporate updates beyond November 2022 limits its applicability for delivering up-to-date clinical guidance. Therefore, although ChatGPT may serve as a supplementary educational tool, its outputs should be interpreted with caution and used in conjunction with professional medical guidance and the most recent evidence-based resources.