Perspectives of students and assessors of standardization in performance-based assessments

Judgment standardization increases the reliability of performance-based assessment but often leads to resistance among stakeholders. In the educational domain, little is known how different components of judgment standardization are perceived by assessors and students. This pre-registered study aimed to address this gap. We applied a mixed-methods approach, using a quantitative between-subjects design (assessors n = 417, undergraduate students n = 400) and qualitative focus groups (n = 20 assessors, n = 20 undergraduate students). The assessors were teachers in higher education. The qualitative data helped to understand reasons for perceptions and to identify themes beyond the perception constructs included in the quantitative part. We found the most positive perceptions for (1) methods with high rating standardization (predefined criteria with definitions and anchored scales) as opposed to low rating standardization (only broadly defined criteria) and for (2) absolute judgment as compared to comparative judgment. Contrary to previous research, our findings suggest that low rating standardization may lead to more resistance for both students and assessors than high rating standardization. Overall, the results highlight that assessors and students value explicit guidelines in determining what should be assessed, and qualitative feedback, to promote student development.