Assessing team performance: a mixed-methods analysis using interprofessional <i>in situ</i> simulation

<p><strong>Introduction:</strong> Optimizing the performance of emergency department (ED) teams impacts patient care, but the utility of current, team-based performance assessment tools to comprehensively measure this impact is underexplored. In this study we aimed to 1) evaluate ED team performance using current team-based assessment tools during an interprofessional in situ simulation and 2) identify characteristics of effective ED teams.</p>\n<p><strong>Methods:</strong> This mixed-methods study employed case study methodology based on a constructivist paradigm. Sixty-three eligible nurses, technicians, pharmacists, and postgraduate year 2-4 emergency medicine residents at a tertiary academic ED participated in a 10-minute in situ simulation of a critically ill patient. Participants self-rated performance using the <em>Team Performance Observation Tool</em> (TPOT) 2.0 and completed a brief demographic form. Two raters independently reviewed simulation videos and rated performance using the TPOT 2.0, <em>Team Emergency Assessment Measure</em> (TEAM), and <em>Ottawa Crisis Resource Management Global Rating Scale</em> (Ottawa GRS). Following simulations, we conducted semi-structured interviews and focus groups with in situ participants. Transcripts were analyzed using thematic analysis.</p>\n<p><strong>Results:</strong> Eighteen team-based simulations took place between January-April 2021. Raters' scores were on the upper end of the tools for the TPOT 2.0 (R1 4.90, SD 0.17; R2 4.53, SD 0.27, IRR [inter-rater reliability] 0.47), TEAM (R1 3.89, SD 0.19; R2 3.58, SD 0.39, IRR 0.73), and Ottawa GRS (R1 6.6, SD 0.56; R2 6.2, SD 0.54, IRR 0.68). We identified six themes from our interview data: team member entrustment; interdependent energy; leadership tone; optimal communication; strategic staffing; and simulation empowering team performance.</p>\n<p><strong>Conclusion:</strong> Current team performance assessment tools insufficiently discriminate among high performing teams in the ED. Emergency department-specific assessments that capture features of entrustability, interdependent energy, and leadership tone may offer a more comprehensive way to assess an individual's contribution to a team's performance.</p>