Evaluating ChatGPT’s Effectiveness in Generating Pharmacogenomic Prescribing Guidelines

Background/Objectives: In precision medicine, patient genetic data is increasingly being used to inform physicians’ prescribing of medication, yet current resources to access gene-drug interaction data are often difficult and unintuitive to use. This study compared the efficacy of ChatGPT and PharmGKB® to extract and generate drug pharmacogenomic prescribing guidelines based on an individual patient’s genetic profile and medication use. Methods: Drug and gene data for 24 patients were input into ChatGPT with a standardized prompt. The recommendations generated were compared to guidelines generated from PharmGKB®, a curated pharmacogenomic knowledge resource that integrates CPIC®, DPWG, and FDA guidance. Evaluation metrics included time to obtain guidelines, the accuracy of the recommendations, and the types of errors made by ChatGPT. Results: ChatGPT output was concordant with PharmGKB® for 148/204 medications (73%), but 124 of these agreements were cases in which neither source identified a guideline. Where a PharmGKB® guideline existed, ChatGPT reproduced it for only 24/53 medications (45%). Errors included omitted medications (n = 41), unsupported recommendations (n = 9), incorrect dosing (n = 4), and missed guidelines (n = 2); ChatGPT also hallucinated 28 medications not on patients’ lists. PharmGKB® averaged 6 min 20.8 s for each patient (SD = 3.5 min), versus 2 min 24.7 s (SD = 32 s) for ChatGPT (p < 0.001). Conclusions:While ChatGPT offers speed and ease of use, its clinical reliability remains limited due to frequent and potentially serious errors. Practical and effective utility of pharmacogenomics is imperative to advancing the use of pharmacogenomics in precision medicine, and more research must be done to elucidate the best tools available for physicians.