Enhancing the defensibility of cut scores: A multi-source evaluation of a modified Angoff standard-setting process in a high-stakes medical examination

Introduction: Standard setting is essential in health professions education, yet establishing credible and defensible cut-off scores remains challenging, particularly when post-standard-setting evaluation is limited. Objective: To apply a systematic, multi-source approach to enhance the validity and defensibility of cut-off scores derived through a modified Angoff procedure in a high-stakes medical examination. Methods: A cross-sectional study was conducted using a final-year MBBS exit examination comprising a Multiple-Choice Question (MCQ) test and an Objective Structured Clinical Examination (OSCE). Fifteen experienced faculty applied the Modified Angoff method to determine initial cut scores. Validity evidence was collected through pre- and post-judgment surveys, psychometric analyses, and acceptability surveys involving students, judges, and medical education experts. Descriptive statistics, standard error adjustments, and independent t-tests were used. Results: Initial cut scores were 64.44/100 (MCQ) and 12.85/20 (OSCE). Judges showed high clarity, confidence, and agreement. Psychometric analysis revealed limited score variability, indicating consistent performance. Cut scores were adjusted upward by one standard error, a theoretically justified approach intended to reduce potential false-positive classifications. Acceptability was high among judges and students, with no significant group differences. Conclusion: Post standard setting evaluation added validity evidence, reinforcing the credibility and acceptability of cut off scores in high stakes examinations.

Authors

Institutions

Publication Details

Journal
GMS journal for medical education
Published
2026-11-16
DOI
https://doi.org/10.3205/zma001889
Primary Topic
Innovations in Medical Education
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Enhancing the defensibility of cut scores: A multi-source evaluation of a modified Angoff standard-setting process in a high-stakes medical examination

Olayinka Rasheed Ibrahim, Anteneh Gadisa Belachew, Equlinet Misganaw Amare, Hiba Abuelgasim Fadlelmoula Abdelrahman et al.
1 links
GMS journal for medical education
Innovations in Medical Education
article

Enhancing the defensibility of cut scores: A multi-source evaluation of a modified Angoff standard-setting process in a high-stakes medical examination

Olayinka Rasheed Ibrahim, Anteneh Gadisa Belachew, Equlinet Misganaw Amare, Hiba Abuelgasim Fadlelmoula Abdelrahman, Biniam Ewnte, Tefera Tadesse, Natalie McCall, Abebe Bekele Zerihun
article en

Abstract

Introduction: Standard setting is essential in health professions education, yet establishing credible and defensible cut-off scores remains challenging, particularly when post-standard-setting evaluation is limited. Objective: To apply a systematic, multi-source approach to enhance the validity and defensibility of cut-off scores derived through a modified Angoff procedure in a high-stakes medical examination. Methods: A cross-sectional study was conducted using a final-year MBBS exit examination comprising a Multiple-Choice Question (MCQ) test and an Objective Structured Clinical Examination (OSCE). Fifteen experienced faculty applied the Modified Angoff method to determine initial cut scores. Validity evidence was collected through pre- and post-judgment surveys, psychometric analyses, and acceptability surveys involving students, judges, and medical education experts. Descriptive statistics, standard error adjustments, and independent t-tests were used. Results: Initial cut scores were 64.44/100 (MCQ) and 12.85/20 (OSCE). Judges showed high clarity, confidence, and agreement. Psychometric analysis revealed limited score variability, indicating consistent performance. Cut scores were adjusted upward by one standard error, a theoretically justified approach intended to reduce potential false-positive classifications. Acceptability was high among judges and students, with no significant group differences. Conclusion: Post standard setting evaluation added validity evidence, reinforcing the credibility and acceptability of cut off scores in high stakes examinations.

GMS journal for medical education
University of Global Health Equity (RW)
Good health and well-being, Quality education
Innovations in Medical Education
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon