Virtual simulation mobile application for teaching medical emergencies: attitudes and behaviours of postgraduate year one doctors and faculty

INTRODUCTION Coronavirus disease 2019 (COVID-19) restrictions reduced real-world experience-based learning, which is crucial for developing clinical reasoning, highlighting the need for alternative training methods.[1] To fill this gap, we explored virtual simulation-based education (SBE), which has been shown to be an effective teaching tool.[2] We identified Full Code©, a suitable proprietary web-based virtual simulation program that trains clinical reasoning, analyses participant performance and provides direct feedback through dynamic clinical scenarios involving virtual patients. The simulation program is customisable to the local context and this allows us to create a tailored case bank and introduce it to learners when the COVID-19 pandemic was at its zenith to complement existing learning activities. This study aimed to assess the attitudes and behaviours of postgraduate year 1 (PGY1) doctors (learners) and the faculty on the use of Full Code©, and evaluate the effectiveness of virtual SBE in teaching common medical emergencies and boosting learners’ confidence when dealing with real-world emergencies. METHODS This prospective, mixed-method, single-centre study involved all PGY1 learners (n = 50) rotating through the National University Hospital, Singapore, between May 2021 and August 2021. Learners were introduced to Full Code© via an open email invitation and encouraged to complete ten standardised cases as a self-learning tool before their preposting objective structured practical examination (OSPE). After the OSPE, participants completed a 20-item questionnaire that we developed to assess the impact of Full Code© on learning and confidence during the OPSE. Thereafter, participants were given 15 new Full Code© cases during their 4-month posting. Data from the questionnaire were analysed using descriptive statistics. Group interviews were conducted with PGY1 learners and the faculty to understand their experience of using Full Code© for learning and teaching medical emergencies. Participation was voluntary. Interview questions were developed and refined through discussion and consensus. Faculty involved in teaching PGY1 doctors with simulation were invited to participate and given a live demonstration of the application before the interviews. Each group interview involved at least two interviewers. Interview data were audio recorded and transcribed by an independent research assistant. NVivo qualitative data analysis software (Lumivero, Denver, CO, USA) was used. Two authors analysed the data using Braun and Clarke (2006) six-step thematic analysis. They independently read the transcribed data to gain familiarity with the context of the data (Step 1), developed initial codes (Step 2) and discussed these codes to ensure credibility and transferability. Similar codes were collated into subthemes (Step 3), which were further examined and combined to form themes (Step 4). The themes were refined for coherence and distinctiveness, accompanied by narratives (Step 5). These were condensed into a report. RESULTS All learners who were graduates from Singapore medical schools responded to the questionnaire. Participant characteristics are summarised in Table 1. All 50 participants used Full Code© and completed between one and ten cases before the OSPE, with a median of nine cases per participant. Twenty-four (48%) participants strongly agreed that the application was easy to use and navigate. Thirty-eight (76%) participants strongly agreed that the cases were of appropriate difficulty, and 26 (52%) participants strongly agreed that the cases were realistic. Twenty-five (50%) participants strongly agreed that the score breakdown post-scenario helped identify knowledge gaps, 25 (50%) participants strongly agreed that the post-scenario discussion helped consolidate learning and 21 (42%) participants strongly agreed that their confidence in managing the physical OSPE scenarios increased. Twenty-eight (56%) participants strongly agreed that virtual SBE helped them translate theory into practice, and 27 (54%) participants strongly agreed that Full Code© is effective in teaching acute management of common medical emergencies for PGY1 [Figure 1].Table 1: Demographic characteristics of learners (N=50).Figure 1: Graph shows learner attitudes towards Full Code© application assessed using Likert scales of agreement. CTSP: called to see patient, OSPE: objective structured practical examination, PGY1: postgraduate year 1The total number of plays, including repeated scenarios, ranged between 1 and 40, with a median of 13 (interquartile range 10.8) cases per participant. Four (8%) participants did not complete any additional cases. Four faculty members and six learners participated in the group interviews. The thematic analysis process yielded various themes and subthemes [Table 2].Table 2: Learner and faculty key themes and subthemes.Learner group For the learner group, five themes were generated based on their experience with Full Code©. From these five themes, 18 subthemes were further generated [Table 1]. Usage of simulation application Full Code© offers accessible low-dose, high-frequency virtual SBE. Learners appreciated its convenience and diverse case repertoire, and found it useful before clinical rotations or call. Increased use of Full Code© also allowed learners to prioritise actions by toggling between segments, which reflect real-world application, where initial assessment and management are highly dependent on patient acuity. Virtual simulations provide a safe practice environment to refine clinical reasoning skills through repetition. Cognitive skills acquisition Full Code© aided in consolidating knowledge and improving medical approach, diagnostic capability and decision-making skills, especially after multiple plays. It facilitated thinking in broad principles and narrowing down of differential diagnoses with history taking and physical examination. Useful technical features of the application Full Code© provided post-scenario scoring and performance feedback. The scoring rubric breaks down scores into tested categories and decision accuracy, such as whether a chosen action is recommended, harmful or unnecessary. This feature emphasised prioritisation, identified knowledge gaps and increased overall confidence. Learners also valued the convenience of the application”s mobile device access, which allowed enhanced learning efficiency. Technical features to improve and incorporate Learners suggested that case difficulty should advance with clinical progress. They also suggested improving the scoring rubric, such as assigning a higher weightage for the correct sequence of actions. They preferred open-ended questions instead of dropdown options to better simulate real scenarios. Overall utility of the application Learners” perception of application utility influences sustained engagement. Learners viewed virtual SBE as an alternative to physical simulation sessions, especially when face-to-face learning is limited. Faculty group For the faculty group, four themes were generated based on their experience with Full Code©. From these four themes, 16 subthemes were further generated [Table 1]. Potential benefits to the learner The faculty believes that virtual simulations can help learners master the fundamentals of advanced-level learning, supplement precourse reading and support individualised independent learning. They felt that learner-perceived benefits of knowledge attainment through virtual SBE will drive greater uptake of this training modality. Useful technical features of the application and suggested improvements The faculty found the hint function useful as it mimicked how nurses engage doctors in patient management. Mastery through repeat plays at the learner’s pace, coupled with post-scenario individualised feedback, allows for flexibility of the application to be used as homework, revision or follow-up. In addition, clear learning objectives should be delineated to both learners and the faculty. Further enhancements can include incorporating onsite educator guidance, live debriefing and team plays with various allied health to reflect the multidisciplinary nature of patient care. However, virtual simulations lack realism, pressure triggers and the ability to assess practical skills as compared to physical simulations. Cases may stagnate while the learner develops clinical maturity, resulting in a ceiling effect. Ultimately, the current technology is not yet capable of replicating the pressure for prioritisation in the setting of simultaneous patient reviews, nor is it adequate to reproduce the true complexity of managing live medical emergencies. Endorsement of the application The decision to endorse the mobile simulation application for learning will be determined by the learners’ experience with the application and supplemented by learners’ feedback. Overall utility of the application Despite the drawbacks, the faculty found the mobile simulation application a useful supplement or alternative to in-person simulation. It is readily accessible, convenient and cost-effective and does not require simulation-trained instructors or a specialised facility to conduct the session. DISCUSSION Learning through direct patient contact has traditionally been the cornerstone of clinical education, but its opportunistic nature, limitations in group size and concerns about patient safety have led to the introduction of SBE. Simulation-based education offers controlled exposure and difficulty of scenarios,[3,4] making it an effective method of learning across different health professional groups.[5,6] Compared to physical simulations, virtual simulations are less resource intensive and costly,[7] allowing for broader learner participation. Key strengths of virtual SBE include bespoke curricula, psychological safety, accessibility, practicality, replicability, scalability[8] and gamification elements that encourage engagement and independent learning.[9,10] Immediate feedback and performance analytics further enhance the learning experience.[9,11] Our study shows that virtual SBE effectively enhances medical knowledge and confidence among junior doctors. Learners appreciate control over their learning pace and the detailed feedback that the application provides, which boost confidence in managing clinical scenarios. For the faculty, the application is time-effective and beneficial for novice learners, especially with clear learning objectives and tailored feedback. Both learners and faculty were receptive to Full Code© as a supplemental educational tool during the pandemic. Full Code© enables the creation of tailored clinical scenarios, enhancing realism and relevance and better preparing learners for clinical application.[12] The low-dose, high-frequency exposure to various scenarios reinforces enactive learning, facilitates clinical knowledge revision and enhances clinical reasoning.[3,13] In addition, standardisation can be leveraged for assessment purposes.[9,11] Virtual SBE has its limitations such as diminished learning benefits with increased clinical experience. Content must be regularly assessed and adapted to maintain relevance. The faculty noted deficits in exposure to practical skills, teamwork, task prioritisation and complex multidisciplinary interactions. Despite these limitations, virtual SBE offers benefits, such as ease of implementation, scalability, patient safety, reduced diagnostic errors, standardised high-fidelity training, reduced costs and logistical burdens, making it a worthwhile endeavour. Real-time post-simulation debriefing is crucial in SBE.[14,15] Although Full Code© offers post-scenario score breakdowns, an educator-guided approach with prebriefing and debriefing could further enhance the learning experience. A limitation of this study is the small sample size for the qualitative component. Despite this, a broad number of themes were gathered from each group. Confirmation bias was minimised through independent transcription of data, while question order and word bias were addressed through rigorous vetting of interview questions by all the authors and by involving trained individuals in interview delivery. In the future, we hope to incorporate virtual SBE into mainstream clinical training and develop a diverse, technologically enhanced educational model. This includes incorporating acute, subacute, and chronic conditions spanning both medical and surgical representations in the application. Scenarios can also be tailored towards enhancing patient safety education with the assimilation of artificial intelligence into the program to improve realism. In conclusion, virtual SBE is an effective tool for enhancing knowledge and increasing confidence in managing medical emergencies for PGY1. Learners and the faculty felt that it confers a pedagogical advantage. Further improvements should encompass an educator-guided approach that outlines case objectives and post-scenario debriefing while facilitating independent learning. Acknowledgement We thank the following individuals: Adj. Prof Malcolm Mahadevan (Designated Institutional Official, National University Health System Residency Program) for his unwavering support in incorporating virtual simulation education in postgraduate education, as well as Mr Michael Mclinn (CEO & Co-founder, Full Code Medical) and Dr James Kimo Takeyesu (Chief Medical Officer & Co-founder, Full Code Medical) for their technical assistance in the usage of Full Code©. Financial support and sponsorship Funding was received from the Residency Operating Expenses, Ministry of Health, Singapore. Conflicts of interest There are no conflicts of interest.