A program evaluation for teaching AI prompt engineering for evidence-based medicine to fourth year medical students

Abstract Background Prompt engineering is an emerging competency in medical education, essential for applying large language models (LLMs) to evidence-based medicine (EBM). This study evaluates an instructional approach designed to introduce medical students to two prompt engineering frameworks using a closed AI-assisted interdisciplinary database to answer clinical questions. Methods Nineteen fourth-year medical students participated in three 45-minute, one-on-one individualized sessions facilitated by faculty. Students were trained to construct AI prompts using PICO (Patient, Intervention, Comparison, Outcome) and CLEAR (Concise, Logical, Explicit, Adaptive, Reflective) frameworks within Scopus AI. Effectiveness was measured through weekly evaluations and end-of-program structured group interviews. Results All students completed the 3-part program. Mean confidence in prompt generation was high throughout the program: 1.16 for week one, 1.32 week two, and 1.05 week three (1 = Strongly Agree, 4 = Strongly Disagree). Students preferred the PICO framework for clinical use, finding it more straightforward and applicable for quick searches than CLEAR. Scopus AI was specifically valued for providing credible, verifiable, and peer-reviewed references compared to other open general AI tools such as ChatGPT. Conclusion Brief, individualized instruction is an effective model for teaching AI literacy and prompt engineering, though scaling requires balancing faculty time with educational value. Utilizing a closed AI system helps mitigate inaccuracies while enhancing student confidence in EBM application. This 3-part one-on-one curriculum provided a viable framework for integrating prompt engineering into fourth-year medical student clinical rotations to support evidence-based, point-of-care clinical decisions.