Enhancing the defensibility of cut scores: A multi-source evaluation of a modified Angoff standard-setting process in a high-stakes medical examination
Introduction: Standard setting is essential in health professions education, yet establishing credible and defensible cut-off scores remains challenging, particularly when post-standard-setting evaluation is limited. Objective: To apply a systematic, multi-source approach to enhance the validity and defensibility of cut-off scores derived through a modified Angoff procedure in a high-stakes medical examination. Methods: A cross-sectional study was conducted using a final-year MBBS exit examination comprising a Multiple-Choice Question (MCQ) test and an Objective Structured Clinical Examination (OSCE). Fifteen experienced faculty applied the Modified Angoff method to determine initial cut scores. Validity evidence was collected through pre- and post-judgment surveys, psychometric analyses, and acceptability surveys involving students, judges, and medical education experts. Descriptive statistics, standard error adjustments, and independent t-tests were used. Results: Initial cut scores were 64.44/100 (MCQ) and 12.85/20 (OSCE). Judges showed high clarity, confidence, and agreement. Psychometric analysis revealed limited score variability, indicating consistent performance. Cut scores were adjusted upward by one standard error, a theoretically justified approach intended to reduce potential false-positive classifications. Acceptability was high among judges and students, with no significant group differences. Conclusion: Post standard setting evaluation added validity evidence, reinforcing the credibility and acceptability of cut off scores in high stakes examinations.
Authors
- Olayinka Rasheed Ibrahim (ORCID: https://orcid.org/0000-0002-2621-6593)
- Anteneh Gadisa Belachew (ORCID: https://orcid.org/0000-0002-9836-2637)
- Equlinet Misganaw Amare (ORCID: https://orcid.org/0000-0002-1115-6911)
- Hiba Abuelgasim Fadlelmoula Abdelrahman (ORCID: https://orcid.org/0000-0002-2568-9620)
- Biniam Ewnte (ORCID: https://orcid.org/0000-0002-3899-2880)
- Tefera Tadesse (ORCID: https://orcid.org/0000-0002-1770-4659)
- Natalie McCall (ORCID: https://orcid.org/0000-0001-9387-4521)
- Abebe Bekele Zerihun
Institutions
- University of Global Health Equity (RW)
Publication Details
- Journal
- GMS journal for medical education
- Published
- 2026-11-16
- DOI
- https://doi.org/10.3205/zma001889
- Primary Topic
- Innovations in Medical Education
- Type
- article
- Field-Weighted Citation Impact
- 0.00