Does the Timing of USMLE Step 1 Affect Performance? A Comparison of Pre- and Post-Clerkship Timing in the Pass/Fail Era

In 2022, Step 1 of the United States Medical Licensing Examination transitioned to pass/fail scoring, removing a major performance-oriented incentive that historically shaped how and why students prepared for the exam. While Step 1 is typically taken before clerkships, some medical schools have shifted the exam to after core clerkships, citing potential benefits for learning and integration. This study examined whether the timing of Step 1 continues to influence performance outcomes in the pass/fail era, using achievement goal theory and self-determination theory as conceptual lenses. Achievement goal theory distinguishes among mastery-approach, performance-approach, and performance-avoidance goals that fluctuate across contexts; self-determination theory emphasizes how assessment environments support autonomous versus controlled forms of motivation. This study compared outcomes at 20 US medical schools that administered Step 1 after clerkships to those at a matched group of schools that administered the exam pre-clerkships. All students matriculated in 2020 and took Step 1 on or after January 26, 2022, when pass/fail scoring began. We matched schools based on average Medical College Admission Test (MCAT) scores to help control for incoming academic ability. We used a resampling procedure with 10,000 replications to estimate average differences in Step 1 and Step 2 Clinical Knowledge (CK) failure rates and scores. Compared to matched pre-clerkship schools, post-clerkship schools had a slightly lower Step 1 failure rate (mean difference: 1.02%, SD = 0.89, CI = 1.00-1.04%). In contrast, Step 2 CK outcomes modestly favored pre-clerkship schools: failure rates were 0.39% lower (SD = 0.21, CI = 0.39-0.4%) and scores were 1.03 points higher (SD = 0.53, CI = 1.02-1.04%) on average. Across all outcomes, differences were small and unlikely to be educationally meaningful. Among students at post-clerkship schools, a shorter interval between Step 1 and Step 2 CK was associated with slightly higher Step 2 CK scores for students with lower MCAT scores, whereas the interval had little impact for students with higher MCAT scores. In the pass/fail context, the timing of Step 1, before or after clerkships, appears to have minimal influence on performance outcomes. These findings suggest that changes to Step 1 scoring and timing alone are unlikely to shift learner behavior; broader assessment and curricular reforms that intentionally foster autonomous, mastery-oriented learning may be needed to support deep learning, well-being, and readiness for practice.