The validity of multiple-true-false and one-best-answer in the final professional undergraduate medical examination

Multiple-True-False (MTF) and One-Best-Answer (OBA) are commonly used multiple-choice question (MCQ) formats in higher education to evaluate cognitive components. While both evaluate factual knowledge, a well-written OBA question can also stimulate problem-solving and knowledge application in clinical case scenarios. This study aimed to evaluate the convergent and predictive validity of MTF and OBA formats in undergraduate medical education assessments. This crosssectional study analysed archival records of 143 students sitting for the 2022 final professional undergraduate medical examinations. SPSS version 24.0 was used to enter and analyse the data. Pearson’s correlation test was used to assess the convergent validity of MTF and OBA, while a linear regression test was used to evaluate predictive validity. Pearson’ s correlation test showed moderate convergent validity (r = 0.25 to 0.6) between the MTF and OBA and other assessment methods. OBA outperformed MTF in predicting key feature question (KFQ) theory assessment (β = 0.40, p < 0.01 vs β = 0.26, p < 0.01), while MTF had stronger predictive validity for clinical components (manned Objective Structured Clinical Examination [OSCE], unmanned OSCE, and modified long case) as compared to OBA (β = 0.43, p < 0.01 vs β = 0.28, p < 0.01). The results are consistent with the literature in that OBA can determine know-how levels compared to MTF. However, a quality improvement exercise must be conducted that focuses on the assessment process of each assessment method, including the assessment blueprint, question structure, examiner calibration, and question vetting. These findings contribute to the enhancement of the quality and validity of assessment practices in medical education