Competency Assessment of New Registered Nurses Using Immersive Virtual Reality

Transitions are vulnerable touchstones in health care. The transition from student nurse to newly licensed registered nurse (NLRN) exposes highly complex patients to novice nurses, which threatens patient safety. Newly licensed registered nurse must rapidly adapt to the demands of clinical practice, navigating various challenges of their new roles, including patient care complexities, interprofessional dynamics, ethical dilemmas, and limited numbers of experienced nurse preceptors.1 There is agreement across the discipline that change is needed for a smoother transition into practice,2,3 to mitigate the high attrition rates among NLRNs, which can directly impact patient safety. Ensuring the preparedness of NLRNs is paramount to promoting safe and effective patient care delivery.Nurse residency programs (NRPs) provide a structured approach to facilitate the transition of NLRNs1,4,5 into practice that may improve knowledge and skills retention if the NRP addresses competency acquisition.6 Nurse residency programs provide a supportive environment where NLRNs receive mentorship, engage in educational activities, and gain exposure to diverse clinical experiences. By fostering professional development and enhancing clinical skills, NRPs aim to mitigate the challenges encountered by NLRNs.Although there are scant reports of traditional simulations used effectively in NRPs,1,7 little is known about the use of immersive virtual reality (IVR) simulation in NRPs. Immersive virtual reality is a technology that aims to transport users into a 3-dimensional digital environment. Users interact with the virtual environment using hand controls and head-mounted displays (HMDs) to provide a personal and immersive experience.8 This immersive quality is crucial to fostering a sense of psychological presence, where users feel as if they inhabit the virtual environment. Presence, the cornerstone of IVR, is the sensation of “being there” in the virtual world that occurs when our motor and perceptual systems seamlessly integrate with the digital environment, mimicking our interactions in the physical world.9 Immersion represents the extent to which IVR can increase this sense of presence by digitally replicating reality, effectively shutting out the physical world.10 Neuroscience studies using functional magnetic resonance imaging have demonstrated that IVR elicits brain responses similar to real-world situations and sometimes even amplifies brain activation. These findings highlight IVR’s physiological impact, warranting exploration of its applications in nursing professional development.Immersive virtual reality has the potential to revolutionize professional education, ushering in a new era of human-computer interaction.9 For nursing, IVR offers immersion in a virtual clinical environment, providing realistic opportunities to practice clinical decision-making, critical thinking, and communication skills within a safe and controlled environment. Additionally, the artificial intelligence (AI)–driven nature of the scenarios supports individualized practice with real-time feedback. The user’s actions are timestamped as tasks and digitally recorded, generating a session log at the end of every scenario for immediate feedback. Users can repeat the IVR scenarios (IVRSs) multiple times without requiring costly resources associated with traditional simulation methods. The data analytics track progression over time, allowing for rich dialogue with experts to reflect on and improve decision-making and clinical judgment skills.Integration of IVR in NRPs can potentially bridge the gap between theoretical knowledge and clinical practice, providing NLRNs with a safe space to refine their competencies with formative feedback. This study investigated the use of the AI-driven data analytics for clinical competence assessment of NLRNs who completed 2 IVRSs using Oxford Medical Simulation (OMS), a commercially available IVR platform.This project used a descriptive design. Ethical approval was obtained from the institutional review board of the university affiliated with the academic medical center, which determined that the project was not human subject testing.The convenience sample was composed of 29 nurses enrolled in a pilot NRP at an academic medical center in the upper Midwest. Each nurse was invited to participate in 2 IVRSs during the second half of a day scheduled for their paid monthly NRP class time.The Creighton Competency Evaluation Instrument (C-CEI) and the American Association of Colleges of Nursing (AACN) Essentials are 2 competency frameworks embedded in the OMS software, allowing for individual and aggregate analysis of performance during each IVRS. The C-CEI is used widely to assess clinical competency.11 Nursing behaviors are divided into 4 subscales of assessment, communication, clinical judgment, and patient safety for analysis of IVR performance by subscale. The AACN Essentials consist of 10 domains that describe broad areas of nursing competence with expected competencies of new nurse graduates.12The IVR session began with a prebriefing that included the OMS scenario learning objectives, a 10-minute orientation to the Meta Oculus Quest 2 (Meta Platforms, Inc) HMD and hand controls, and a 10-minute orientation to the OMS software to ensure that participants could access the resources within the virtual hospital. Each participant was then distributed an HMD and hand controls to independently complete IVRS 1, which focused on the care of a patient admitted to an adult medical-surgical inpatient unit with pyelonephritis. Immediately afterward, participants engaged in a 40-minute guided reflective debriefing in groups of 13 and 14. The debriefings were led by instructors who had each received a 1-hour recorded debriefing training focused on how to use Socratic questioning in a reflective dialogue and had previously led debriefings following an IVRS. After a short break, participants independently completed IVRS 2, which focused on nursing care of 2 patients, one of which was the now-stable patient for whom they provided care in the first scenario. A second patient was a direct admit from a provider’s office experiencing high blood glucose levels due to a lack of funds for insulin. Immediately following IVRS 2, all participants engaged in a reflective debriefing in 2 groups.Twenty-seven nurses completed IVRS 1 and 2 during an NRP class day. Each participant began employment as an NLRN between August and October 2023, representing various clinical departments, predominantly inpatient care settings. the Tableillustrates the distribution of participants across clinical departments.The OMS data analytics generated by the users during the 2 scenarios were deidentified and analyzed aggregately by selecting each competency framework in the OMS dashboard. The data were first analyzed according to the C-CEI competency framework. For IVRS 1, participants aggregately scored 66% for the total C-CEI. The subscale scores were 75% (assessment), 69% (clinical judgment), 59% (communication), and 59% (patient safety). For IVRS 2, the aggregate score was 61% for the total scale, while the subscales were 67% (assessment), 61% (clinical judgment), 63% (communication), and 52% (patient safety). See Figure 1for a comparison of the 2 scenarios by C-CEI subscale.Next, data were analyzed using the AACN Essentials domains in the OMS dashboard (Figure 2). For IVRS 1, participants scored between 55% (interprofessional) and 100% (scholarship). For IVRS 2, participants scored between 43% (scholarship) and 65% (interprofessional), with scores in the 60% range for all other domains.The findings of this pilot demonstrated that the data generated during the NLRNs’ completion of 2 IVRSs can be quickly analyzed using 2 embedded nursing competency frameworks without the bias or subjectivity inherent to clinical assessment. The AI-driven nature of the IVR data analytics enables individualized assessment of the actions and decisions made in a scenario, demonstrating the potential for using IVR to assess nursing competencies. The results of the data analytics demonstrated varying levels of competence and indicated areas for improvement.The possibility of accurate clinical competence assessment of nurses’ IVRS-generated data is immense, as it eliminates any potential bias from raters. However, realizing this potential necessitates precise programming of the IVRS to clearly reflect actions and decisions that correlate with each nursing competency. This requires validating how each virtual task is encoded within IVR software to ensure accurate evaluation of each competency and validity of the scoring process. In addition, further research should investigate the long-term impact of IVR training on NLRN competence and explore the extent to which performance in virtual environments correlates with and translates into real-world patient care delivery.The findings of this pilot study are limited by the small sample size and may not be generalizable to all populations.Immersive virtual reality offers a novel approach to immerse NLRNs in realistic patient care scenarios and assess clinical competence using IVR-generated data analytics. The AI-driven system offers a promising method for refining nursing competencies with individualized feedback.