Equity in assessment: an examination of multiple-choice question tests

Equity and fairness are essential elements of acceptability, one of the Ottawa Criteria for good evaluation. The similarity of results from multiple-choice question (MCQ) tests developed under the same test blueprint for comparable groups can serve as empirical evidence of equity in assessment. This study aimed to examine whether MCQ tests prepared for repeated clerkship groups within one academic year demonstrated psychometric equivalence, focusing on item difficulty (DIF I) and discrimination (DI) indices as indicators of instrumental equity. Five MCQ tests administered at the end of the gynecology–urology internship during the 2022–2023 academic year were analyzed. All tests were developed in adherence to the same blueprint, ensuring identical content representation and cognitive level distribution. For each test, mean score, Kuder–Richardson 20 (KR-20) reliability coefficient, and standard error of measurement (SEM) were calculated. At the item level, difficulty index (DIF I), discrimination index (DI), and nonfunctional distractor (NFD) count were determined. Pairwise equivalence tests were conducted across five exams (ten comparisons) using 90% confidence intervals with predefined equivalence margins (Δ = ±0.10). Equivalence was concluded when the entire CI of the mean difference fell within these limits. Mean test scores ranged between 69 and 74; KR-20 values between 0.85 and 0.90; and SEM values between 3.44 and 3.64. While the mean scores differed significantly between groups (F = 3.217, p = 0.013, η² = 0.039), the differences were of low to moderate magnitude. All ten pairwise comparisons for both DIF I and DI met the predefined equivalence criterion, indicating psychometric similarity in item difficulty and discrimination across tests. However, none of the NFD comparisons met the equivalence criterion, suggesting minor fluctuations in distractor performance, likely due to technical variation rather than systematic bias. The fulfillment of equivalence criteria for DIF I and DI across all comparisons provides empirical evidence that the exams shared comparable psychometric properties. This consistency supports the notion of instrumental equity, reflecting fairness through the operational equivalence of assessment tools. Although full equivalence was not achieved for NFD rates, the overall psychometric consistency across tests indicates that student cohorts were evaluated with examinations of equivalent difficulty and discriminative capacity. Adhering to the test blueprint ensured balanced content representation, contributing to fairness and reliability in assessment. Routine monitoring of test-level and item-level metrics can serve as a practical approach to maintaining equity in medical education assessments.