Comparing Text-Based and Virtual Reality–Based Approaches for AI-Enhanced Instruction in Functional Communication Training: Exploratory Randomized Trial

Abstract Background Training practitioners to implement functional communication training (FCT) is resource intensive. Emerging technologies, such as virtual reality (VR) and chatbots, may deliver training at scale. Their comparative effectiveness and whether skills taught in one modality transfer to another remain uncertain. Objective This exploratory study evaluated the effect of multicomponent training packages featuring either VR or a chatbot on (1) posttraining performance on FCT implementation probes administered in both formats and (2) changes in participants’ self-perceived knowledge, confidence, and usefulness. Methods A convenience sample of 23 university students (mean age=24 y; 15/23, 65% female) were recruited through classroom announcements and peer referrals. Eligible participants (at least 18 years of age, available to attend trainings, and who had no history of discomfort associated with VR) were randomly assigned to either VR (n=12) or chatbot (n=11) training. Groups did not differ significantly across demographic characteristics. In-person trainings consisted of video-recorded instruction and rehearsal of steps in VR or with the chatbot during a single session. Approximate times of sessions, including tutorials and assessments, were 45 minutes (VR) and 55 minutes (chatbot). All participants’ performance on FCT procedures was measured using 2 formats: a VR-based observed mastery probe (OMP) and a text-based probe (TBP). Participants also completed a survey measuring their self-perceptions. Results The number of FCT steps participants performed correctly (of 18) was analyzed using a binomial generalized linear mixed model, with random intercepts for participant and fixed effects of training group (chatbot vs VR), time (pretest vs posttest), and probe modality (OMP vs TBP); α =.05. Results indicated a main effect of time (odds ratio [OR] 36.09, 95% CI 20.75‐62.75; P <.001). A significant 3-way interaction (time × probe modality × training group) indicated that the pattern of skill transfer differed by training condition and probe type (OR 6.10, 95% CI 2.08‐17.86; P< .001). At posttest, the chatbot group significantly outperformed the VR group on the TBP (OR 4.14, 95% CI 1.20‐14.24; P= .049), completing, on average, 4 more steps in the procedure (average marginal effect 0.22, 95% CI 0.04‐0.41). Differences between groups on the OMP were not significant (OR 0.61, 95% CI 0.18‐2.07; P =.43). Across all self-report measures, there was no differential improvement by training condition at posttest. Conclusions This study provides the first comparison of chatbot- and VR-based training for FCT. This extends prior work that compared VR against didactic instruction or assessed performance using a single modality. Findings contribute to the field by supporting the viability of text-based chatbots in training and underscoring the importance of the alignment between training and assessment modalities. Chatbot-enhanced training systems may extend access to evidence-based techniques, such as FCT, in areas with limited training resources. More work is needed to confirm effects and demonstrate applicability to practice.