Impact of the Away Rotation on Program Match Rates in Era of Virtual Residency Recruitment

work has demonstrated that end-of-shift assessments often overestimate proficiency.There is a lack of literature regarding the role of consistent portfolio assessments by the Clinical Competency Committee (CCC) in real-time to establish milestone rankings expediently.Objectives: We developed an assessment tool to generate improved real-time milestone-based feedback for EM residents and program leadership.Methods: We developed an end-of-block (EOB) assessment to be completed at the end of every block by core faculty trained in milestone assessment in the 2022-2023 academic year at a single institution.This assessment evaluated 20 of the 22 current sub-competencies in EM.Every core faculty member assessed each resident they clinically worked with in the preceding block and assessed their performance in simulated settings, didactic presentations, and other interactions.The sub-competencies were averaged and reported to the CCC as a baseline dataset.We analyzed differences in the average of the assessment compared to the ultimate milestone rating assigned by the CCC.Results: Faculty completed 399 EOB assessments, resulting in 960 independent milestone averages.The intraclass coefficient was 0.976 for single measures and 0.9888 for average measures.These correlations were strongest in postgraduate year (PGY)-3 class but less for PGY-2 residents.Conclusions: While this study was conducted at a single site and in a specific specialty, which may limit the generalizability of the results, our results indicate that a CCC can utilize EOB assessments as an assessment tool for milestone determinations.The shorter interval assessment allows for immediate review of resident performance and can be used in real-time by residents to track their progression through the milestones.