A Large Language Model-Driven Embodied Conversational Agent for Patient Education in Virtual Reality: Mixed Methods Usability Study (Preprint)

BACKGROUND Clinicians routinely distribute printed patient education materials at the point of care. Tutorials covering all relevant information are rarely feasible within clinical time constraints and ongoing educational support is commonly needed. Patients reading these materials on their own cannot ask questions or have passages explained, and a pamphlet does not adjust to a patient’s educational needs. Embodied conversational agents (ECAs) powered by large language models could take on the role of a clinician or health educator, guiding a patient through educational materials they were given. Virtual reality (VR) additionally affords spatial grounding of anatomical references and use of attentional cues from the patient, such as gaze and pointing, as human educators do while teaching. Whether patients find such a system usable and acceptable in VR is unknown. OBJECTIVE We evaluated an agentic workflow that transforms static patient education pamphlets into structured learning sessions delivered by an ECA in VR. We conducted a formative evaluation of the usability and acceptability of the VR learning experience, including pre-post knowledge change. METHODS We conducted a mixed methods usability study. Healthy adult volunteers (N=22 enrolled; N=21 analyzed) completed a VR learning session (mean duration 18.0 minutes, SD 3.5) in which the ECA delivered a multimodal walkthrough of standardized patient education materials on colonoscopy, appendectomy, or kidney cancer, supported open-ended questions, and reinforced understanding through summaries and brief multiple-choice review questions (flashcards). Measures included overall satisfaction, perceived quality of the education material using an adapted version of the Patient Education Materials Assessment Tool, pre- and post-session knowledge tests, and semi-structured interviews. RESULTS Satisfaction with the VR learning session was positive (mean 6.35, SD 0.92 on a 7-point scale, significantly above the scale midpoint), as was perceived content quality (mean 6.21, SD 0.88 on a 7-point scale). Knowledge test scores increased significantly from mean 23.45 (SD 8.82) before the session to 34.90 (SD 7.56) after, on a 0-50 scale (paired t test, t20=5.40; P<.001). Interview themes included immersion and spatial visualization of anatomy, the agent’s supportive manner, and interactive feedback. Participants also reported headset weight and discomfort, unfamiliarity with VR controls, and problems with the ECA’s synthesized voice. CONCLUSIONS In a single-session formative evaluation with healthy, predominantly highly educated volunteers, an ECA powered by a large language model delivering a guided walk-through of patient education material in VR was usable and well accepted, and knowledge scores increased significantly from pre- to post-session. These findings suggest that patient education may be augmented with an embodied agent, though specific hardware and speech-synthesis barriers need additional adaptation before evaluation with clinical populations.