AI-driven multimodal simulation for psychiatric clinical education: conceptual framework and preliminary validation of the cognitive core

BACKGROUND: The global shortage of psychiatrists limits learners' exposure to authentic patient encounters. Simulation can support scalable deliberate practice; however, standard virtual patients may under-represent non-verbal and paralinguistic cues that are central to psychiatric assessment. OBJECTIVES: This study adopted a multiphase approach to (1) quantify baseline training gaps, (2) evaluate the educational impact of a Phase 1 "AI-Mentor" prototype (a GPT-4o cognitive core with standard audio-visual features), and (3) identify residual limitations in relational competence to justify future high-fidelity multimodal integration. METHODS: A multiphase mixed-methods study was conducted at Chongqing Medical University. Phase 1 involved a baseline needs assessment survey (N = 68) of undergraduate psychiatry students. Phase 2 comprised a pilot intervention where a cohort engaged with LLM-driven virtual patients. System usability and technology acceptance were evaluated using the Technology Acceptance Model (TAM) (N = 43). Within this cohort, clinical competence was assessed for a robust paired sub-sample (n = 32) using a pre/post-test interview-focused Mini-Clinical Evaluation Exercise (Mini-CEX) rated by senior psychiatrists. RESULTS: Phase 1 revealed that 88% of students struggled to master non-verbal cues through traditional methods. In Phase 2, the Mini-CEX evaluations (n = 32) demonstrated significant pre-to-posttest improvements in overall performance (mean change: 0.82; P < 0.01; Cohen's d = 0.57) and reasoning-aligned domains, notably clinical judgment/diagnostic formulation (P < 0.01; Cohen d = 0.96). However, improvements in humanistic qualities/rapport were much smaller and not statistically significant (P = 0.256), highlighting a persistent humanistic gap. Despite this, TAM evaluations (N = 43) revealed exceptionally high perceived usefulness (Mean = 4.14/5) and perceived ease of use (Mean = 4.15/5), indicating strong learner acceptance of the cognitive core. CONCLUSIONS: The Phase 1 AI-Mentor prototype effectively strengthens diagnostic reasoning and structured interviewing in early psychiatric training. However, the limited improvement in relational competence confirms that standard audio-visual cues are insufficient. This empirical finding establishes the necessity for deeper multimodal perception in subsequent system development.