The role of in situ simulations in the assessment of pediatric anesthesiologists’ non-technical skills

Non-technical skills (NTS) are essential for the safe delivery of anesthesia care, and in situ simulation offers a realistic and immersive environment for developing these competencies. Evidence on whether learning achieved in simulation transfers to real clinical performance particularly among experienced clinicians remains limited. This study aimed to assess pediatric anesthesiologists’ non-technical skills using in situ simulation and to examine the extent to which performance observed during simulation reflects baseline performance in the operating room. This observational pre–post study included eight pediatric anesthesiologists who participated in eight in situ team simulation sessions. Real-case observations were conducted before (n = 32) and after (n = 32) the simulation intervention. Simulation video recordings were used to compare anesthesiologists’ baseline NTS performance in simulated scenarios with their pre-intervention performance in real clinical settings. NTS were assessed using the Turkish version of the ANTSdk scale (ANTSdkt). Data were analyzed using repeated-measures ANOVA for pre–post comparisons. Descriptive statistics were used to summarize ANTSdkt scores observed during simulation and real clinical cases. Inter-rater reliability was assessed using the Intraclass Correlation Coefficient (ICC). No statistically significant improvement was observed in any NTS category or in overall scores following the in-situ simulation training (p > 0.05). Descriptive analysis showed that NTS performance remained stable across both clinical and simulated environments, with an overall score mean score of 4.72 (± 1.27) before the intervention and 5.18 (± 0.88) during the simulation sessions. Inter-rater reliability for simulated cases was good (ICC = 0.79). This study focused on the assessment of pediatric anesthesiologists’ non-technical skills rather than their longitudinal development. Accordingly, a single in-situ simulation session was not expected to result in measurable improvement in non-technical skills, likely reflecting the high baseline performance of experienced clinicians, while allowing observation of skills that appeared consistent across simulated and real clinical settings. The observed alignment between non-technical skills demonstrated during simulation and those exhibited in routine clinical practice suggests that in-situ simulation provides a realistic and informative context for observing clinician behavior in authentic settings.