It AIn’t Perfect: Artificial Intelligence Errors in Allergy

ChatGPT, an online artificial intelligence (AI)-driven language processor, was released in November 2022. Such systems interpret written language, analyze information, and generate responses to questions. The purpose of this investigation was to explore the function and accuracy of how well ChatGPT (version 3.5) responded to a common drug allergy question.The authors entered the prompt, “Write a 500-word editorial on whether it is safe to give cefazolin to a patient with a history of penicillin allergy.”.The chatbot produced a well written and credible 568-word article, citing 2 studies on the safety of cephalosporin use in patients with penicillin allergy and included the authors’ “conclusions” from one of them. ChatGPT was then asked to provide full journal references for those 2 studies. Although the citations seemed accurate, neither the references cited nor the studies described were found in a manual search of the journals named. A search for the digital object identifier of the citations disclosed journal articles about totally unrelated topics. Finally, a manual search for the titles of the articles cited by the chatbot identified articles by entirely different authors.ChatGPT is not completely reliable, with reasoning errors and fabrication of references. The errors highlight the need to “trust but verify” AI-produced responses. The authors call for all medical journals to have policies requiring declaration of AI-generated contributions. Continued discussions regarding the role of AI in medicine are essential.The use of AI has led to false case citations in legal briefs, is a potential form of “plagiarism 2.0,” could aid test-takers cheating on exams, and even might create a risk for medical providers to make incorrect diagnoses or recommend inappropriate therapies. This small study only looked to answer one question related to allergy, so we cannot extrapolate to all queries we may have. Nonetheless, it reminds us of the vagaries of AI.Although AI is a powerful and potentially useful tool, users must still be aware that any product so generated should be reviewed and double-checked for accuracy. Finally, to answer the question submitted to ChatGPT in this study, we refer the reader to this statement: “… for those rare patients with a history of anaphylaxis to penicillin, a noncross- reactive cephalosporin (eg, cefazolin) can be administered routinely without testing.” (Khan DA, et al. Drug allergy: a 2022 practice parameter update. J Allergy Clin Immunol. 2022;150(6):1333–1392.)