Systematically Reviewing the Rigour of Immersive Virtual Reality Research in <scp>STEM</scp> Education: A Deep Dive Into Threats to Internal Validity

ABSTRACT Background There have been many initiatives focused on implementing IVR in the classroom to either replace or supplement conventional instruction. The efficacy of these initiatives is often informed by IVR media comparison studies that examine the learning outcomes of students who learn academic content in IVR versus conventional media. However, the current methodological rigour of research on the use of IVR for learning in STEM education has not yet been established to ensure recommendations concerning its use in education are rooted in rigorous research studies. Objective The present review aimed to fill this gap in the literature by evaluating published journal articles, conference proceedings, and dissertations (between the years 2013 and 2023) related to IVR comparison studies in STEM education. Method The 44 articles were evaluated with respect to 13 internal validity controls (i.e., research design, justified sample size, group equivalence, attrition equivalence, operational definitions provided, matched dosage, matched timeframe, matched content, implementation fidelity, practice equivalence with dependent measures [ DMs ], equivalent DMs , interrater agreement for DM scoring and technical adequacy of DMs ). Results Results indicated there were no articles that explicitly met all 13 internal validity controls; the average number of control issues was 5.36. When articles were not penalised for missing information, 21 articles adhered to all internal validity controls. As can be seen when comparing these findings, the lack of methodological information was a glaring problem. Conclusion These findings indicate that there are both methodological problems and reporting problems in IVR media comparison research that need to be addressed to advance future IVR research. Recommendations for productively moving forward are discussed.