Lecturers’ question difficulty estimation: measuring difference between table of specification test and students’ results of human sciences programme courses

Abstract: Questions in educational assessments must be valid, reliable and fair to students to ensure minimal error in measuring student learning outcomes. One of the methods to ensure fairness of questions used in educational assessments is that to plan for appropriate proportion of questions of different difficulties. Lecturers of Human Sciences programme in the Department of Human Sciences, Centre for Foundation Studies, International Islamic University Malaysia, design Table of Specification (TST) to distribute different difficulties of questions for an educational assessment and lecturers estimate the questions’ difficulty based on expert opinion. Currently, the accuracy of these estimations has yet to be verified as part of the department’s assessment practices. This situation impedes the continual quality improvement of the questions. The purpose of this paper is to verify the accuracy of lecturers’ estimation through measuring the difference between the question difficulty stated in the TST and the difficulty stated in the results of item analysis from students’ results. Pen and paper tests of all five courses in the Human Sciences programme were selected and the comparison of question difficulty between TST and item analysis from students’ results was made. Results suggested that there is a difference between question difficulty stated in TST and item analysis from students’ results. The explanation of results and the limitation of the study are described in the discussion section. In conclusion, the paper indicated that there should be a review of TST and the questions to improve the quality of the educational assessment. Keywords: Educational Assessment, Table of Specification Test, Question Difficulty Estimation, Question Difficulty Based on Students’ Results, Assessment Fairness