AI-Simulated Patients for Training Shared Decision-Making: Feasibility Study in Medical Education

BACKGROUND: Shared decision-making (SDM) is a key element of patient-centered care; however, opportunities for structured and scalable SDM training remain limited in both medical education and clinical practice. Advances in AI have enabled chatbot-based simulations that may support repeated practice and provide automated feedback on SDM-related communication behaviors. OBJECTIVE: This study aimed to (1) evaluate the agreement between AI-generated and human ratings of SDM performance, (2) compare SDM-related performance and attitudes between medical students and physicians, and (3) explore changes in communicative self-efficacy (CSE) following chatbot-based SDM practice. METHODS: We conducted a feasibility study using a pre-post mixed methods design. Medical students and physicians practiced SDM in a 20-minute chat-based interaction with an AI-simulated patient, followed by automated feedback on SDM performance. Prior to the interaction, participants' attitudes toward SDM (IcanSDM) and their CSE (Self-Efficacy in Patient-Centeredness Questionnaire [SEPCQ-24-GER]) were assessed; CSE was reevaluated after the intervention. SDM performance was measured using the validated Observing Patient Involvement in Decision Making (OPTION-12) scale, with the AI evaluating participants' performance from the patient's perspective. These AI-generated ratings were then compared with human ratings to evaluate agreement and rating quality. To further investigate discrepancies between AI and human ratings, a qualitative analysis of selected cases was conducted. Additional measures included demographics, perceived authenticity, and benefits of the interaction. Quantitative analyses included group comparisons, pre-post analyses, and psychometric evaluation of the OPTION-12 scale in an AI-mediated setting. The study was preregistered on the Open Science Framework. RESULTS: A total of 58 participants were included in the analysis, 22 (37.9%) of whom were students, 34 (58.2%) were physicians, and 2 did not declare. Both groups reported similarly positive attitudes toward SDM, but medical students (n=22; mean 28.67, SD 4.32) achieved higher OPTION-12 performance scores than physicians (n=34; mean 22.79, SD 8.14). CSE showed a slight decrease following the intervention. Agreement between AI-generated and human OPTION-12 total scores was high (intraclass correlation coefficient=0.86, df=2,1; P<.001), although AI ratings were systematically higher. The OPTION-12 scale in the AI setting demonstrated good internal consistency (Cronbach α=0.88; mean interitem correlation r=0.43). Participants perceived the AI-simulated interaction as authentic and potentially useful for SDM training. CONCLUSIONS: This feasibility study provides preliminary and exploratory insights into the use of AI-simulated patients for SDM training in medical education. The findings contribute to the contextual validation of the OPTION-12 scale in AI-mediated consultations and provide valuable insights into participants' attitudes toward SDM. However, while findings suggest potential for supporting communication skills development and structured feedback, conclusions are limited by the study design and sample size. Further research with larger samples and controlled designs is needed to establish effectiveness and generalizability.