Generative artificial intelligence for patient drug information: a new framework for content construction

Abstract Introduction Simplifying medication information, using plain language, and improving layout structure can enhance patient comprehension.[1] Generative artificial intelligence (Gen AI) has growing applications in pharmaceutical care, yet the quality and efficiency of its output varies with the chosen models and prompting strategies.[2] Aim To compare multiple large language models (LLMs) and prompting strategies for generating patient drug information and determine an optimal Gen-AI–based workflow. Methods Three medications—pregabalin capsules, acetaminophen extended-release tablets, and levofloxacin tablets—were selected. Eight LLMs (ChatGPT-4o, DeepSeek-R1, Grok 3, and Gemini 2.5 Flash, Doubao, Doubao Deep Thinking, Kimi, Kimi Long Thinking k1.5,) with five prompt strategies (Zero-Shot, Zero-Chain-of-Thought [Zero-CoT], Tree-of-Thought [ToT], Zero-Shot & Few-Shot [ZS&FS], and Zero-CoT & Few-Shot [ZC&FS] yielding 40 combinations. Two pharmacists independently evaluated each text across six dimensions selected aligned with patient-education frameworks and Gen-AI–specific risk considerations: scientific accuracy, comprehensiveness, hallucination, readability, reading time, and accessibility. The first five used a five-point Likert scale using predefined rules or adapted from validated assessment tools; accessibility was scored by the number of human-AI interactions (maximum 3). Quality was defined as the mean total score across dimensions of nine outputs per combination, and stability as the standard deviation (SD). Inter-rater reliability was assessed using a two-way random-effects intraclass correlation coefficient (ICC). Discrepancies were resolved through discussion. The optimal combination was identified through integrated analysis of quality and stability. Results A total of 360 outputs were generated. Inter-rater reliability was excellent (ICC = 0.93; 95% CI: 0.88–0.97; p < 0.001). There are significant differences in the quality and stability. Quality scores ranged from 19.16 (Grok 3 + ZC&FS) to 26.15 (DeepSeek-R1 + ZS&FS). Stability (SD) ranged from 0.31 (Gemini 2.5 Flash + ZC&FS) to 7.42 (ChatGPT + Zero-Shot). DeepSeek-R1 consistently ranked among the highest-performing models, and its combination with ZS&FS was identified as the optimal workflow when balanced between quality and stability. Rankings are shown in Fig. 1. Conclusion This study provides a structured, multidimensional evaluation of LLMs and prompting strategies and has identified an optimal framework for generating patient drug information using generative AI. With pharmacist review, AI-generated materials can support accurate, readable, and accessible medication information, promoting rational drug use. Limitations include the lack of patient usability testing and the evolving nature of LLM capabilities.