Change in Medical Student Preparation for USMLE Step 1 after the Transition to Pass/Fail Reporting: A Retrospective Cohort Study in an International Medical Program

Abstract In 2022, the USMLE Step 1 transitioned to pass/fail reporting, coinciding with a national decline in first-time pass rate. Study behaviors, particularly engagement with retrieval-based practice, are strong predictors of Step 1 readiness. This study examined changes in study behavior before and after the pass/fail transition at a single institution requiring a passing score on an NBME Comprehensive Basic Science Examination (CBSE) as part of a Step 1 preparation course. This retrospective study analyzed UWorld question bank engagement among second-year medical students at the Ochsner Health/University of Queensland Medical School. First-time Step 1 pass rates declined from 97% to 93% between pre-change (2019–2021) and post-change cohorts (2022–2024). The post-change cohort completed 26% more UWorld questions early in their preparation period but reduced question usage by 36% after achieving a passing CBSE score, whereas pre-change students increased usage in the months following the CBSE prior to sitting Step 1. Reduced retrieval-based study activity following CBSE success may contribute to declining Step 1 performance in the pass/fail era. These findings suggest that readiness assessments may influence subsequent student study behaviors, highlighting the need for medical programs to encourage sustained practice engagement after assessment milestones.