Analyzing the quality of multiple-choice questions in microbiology for second-year MBBS students

Abstract Background and Objective: According to the 2023 Graduate Medical Education Regulations by the National Medical Commission for competency-based medical education in the MBBS curriculum, 20% of assessment questions are now multiple-choice questions (MCQs) in formative and summative evaluations. This study aims to evaluate the quality of MCQs in microbiology assessment for second-year MBBS students and compare different methods for calculating the difficulty index, discrimination index (DI), and distractor effectiveness (DE) of the items. Materials and Methods: This study analyzed 20 MCQs, including 20 correct answers (keys) and 60 distractors, from an internal microbiology assessment of 150 students. The items were evaluated for difficulty index (DIF I), Discrimination Index (DI), and Distractor Effectiveness (DE) using three methods: (1) entire group analysis, (2) grouping by the upper third and lower third of students (50 in each), and (3) selecting the top and bottom 27% of students. Results: The difficulty index was calculated using the whole group, upper-third vs. lower-third, and top vs. bottom 27%. Results indicated that the whole group and upper-third/lower-third methods provided similar accuracy, while the top vs. bottom 27% selection method tended to underestimate difficulty levels. The DI, useful for distinguishing between high and low performers, was best calculated using the top and bottom 27% of students, as whole-group analysis was found irrelevant. Two methods were used to measure DE: whole-group and upper-third/lower-third. However, neither approach showed statistically significant differences ( P < 0.05). Conclusion: This study highlights the importance of selecting quality MCQs with moderate difficulty, high discriminative power, and effective distractors to accurately differentiate student performance. Findings suggest that using the top and bottom 27% group is more effective than using the upper-third/lower-third group for analysis of the items.