Exploring communication self-efficacy and artificial intelligence generated assessment tools in primary care education

Abstract Introduction Effective doctor–patient communication improves patient outcomes, satisfaction, and physician well-being. Despite its inclusion in medical curricula, valid assessment of communication self-efficacy remains challenging. Self-efficacy is a possible predictor of performance and is commonly used in the evaluation of competencies. This study aims to (1) map communication self-efficacy using an adapted version of the self-efficacy scale (NGSES) in the context of communication training, and (2) examine the potential of AI-generated assessments as a scalable approach for context-specific self-efficacy assessment. Methods In this exploratory validation study, medical students and general practitioners (GPs), divided into young GPs and specialists participated in a one-day workshop on communication. Self-efficacy was measured pre- and post-intervention (Q1 vs. Q2), using an adapted version of the NGSES and an AI-based scale (ChatGPT-3.5 (OpenAI) based model, assessed February 2023). Internal consistency was assessed using Cronbach’s α. Levels of self-efficacy were compared both before and after the intervention, as well as between the groups (medical students, young GPs and specialists). Statistical analyses included Wilcoxon tests, t-tests, and correlation analyses. Results Communication self-efficacy as perceived by participants improved significantly following the workshop (W = 695 , p< .001 , b R = 0.782) . Post-intervention, medical students self-reported the initial self-efficacy level of specialists (Q1 specialists: Md (IQR) = 2.00 (0.438); Q2 students: Md (IQR) = 2.25 (0.50). Young GPs showed post-training scores comparable to those of specialists (Q2 young GPs: Md (IQR) = 1.94 (0.75); specialists: Md (IQR) = 2.00 (0.25)). Female participants initially reported significantly lower self-efficacy than males (female: Md (IQR) = 2.50 (0.531); male: Md (IQR) = 2.13 (0.375); U = 178,p= .0015, b R = 0.462 ). The AI-based scale demonstrated acceptable internal consistency (Cronbach’s α = 0.811), within a similar range comparable to the adapted NGSES (α = 0.833). Conclusion Our results suggest that communication self-efficacy increased even following a brief training intervention. Professional experience is correlated with higher initial self-efficacy. AI-generated assessment tools may represent a feasible preliminary approach for developing context-specific questionnaires, but further validation is required. The findings should therefore be interpreted as preliminary evidence regarding the feasibility of AI-assisted scale development.