LLM-Based Psychiatric Interview Simulation: Technical Development and Pilot Expert-Based Content Validation of a Voice Prototype

Abstract Objective This study describes the technical development and a pilot expert-based content validation of an interactive voice prototype, using the GPT-4o model, for teaching psychiatric semiology. It explores the potential of large language models (LLMs) to generate real-time feedback and address challenges in acquiring complex clinical competencies. Methods Four psychiatric patient personas (major depressive disorder, bipolar disorder—manic episode, schizophrenia, and attention-deficit/hyperactivity disorder) were developed through a structured iterative process. A subject matter expert (SME) conducted a structured heuristic evaluation to assess clinical fidelity, consistency, and vocal expressiveness across the four scenarios. Quantitative data (Likert scale scores) and qualitative feedback were mapped to identified technical and semiological challenges. Results GPT-4o simulated diverse personas with distinct profiles. The SME reported high potential pedagogical value for practicing interviewing mechanics. However, challenges included stereotyped clinical presentations and platform-imposed content restrictions. Conclusions This pilot study establishes the technical feasibility of voice-based LLM simulations. While offering a promising complementary tool, current limitations in multimodal realism and the need for rigorous faculty supervision suggest that the tool is best suited for formative practice rather than high-stakes assessment.