Fit for purpose? Evaluating multiple-choice question quality in E-learning for emergency primary healthcare

Objective Emergency medicine healthcare professionals need to be continuously updated and trained in emergency medicine. Continuing professional development (CPD) courses are frequently offered but are seldom quality assured by other means than learner satisfaction evaluations or pre–post-test of knowledge retention. This study explores the quality of multiple choice questions (MCQs) used in a mandatory cross-disciplinary Norwegian CPD e-learning course in emergency medicine primary care.Design Cross-sectional study design.Setting We extracted MCQ assessment data from the e-learning platform ‘Oppvakt’ at National Centre for Emergency Primary Health Care (NKLM).Subjects Physicians and nurses working in the Out-of-Hour service in Norway.Main outcome measures Results from the MCQ exam and information about the participants’ professional role was extracted. Statistical analyses were performed to determine the level of difficulty of the MCQ items and their discriminatory power.Results A total of 2310 physicians and 1547 nurses completed the e-learning module, and 79% of the physicians and 67% of the nurses passed their first exam attempt. The physicians and nurses had an average of 85% and 81% correct answers, respectively. More than half of the MCQ items tested facts and knowledge retrieval, and not knowledge application. 70% of the items were found to be ‘easy’ or ‘too easy’. A third of the items had a low level of discrimination.Conclusions Our results indicate that there is room for improvement of the MCQ exam. Overall, the items were too easy and lacked the ability to discriminate between high and low-performance participants.