Rubricizing the assessment practice: A systematic review and meta-analysis of rubrics in rater-mediated assessment of language interpreting

Assessment rubrics have increasingly been developed and deployed to evaluate the quality of language interpreting, yet understanding of rubric-based interpreting assessment remains limited. This systematic review aims to: (a) catalog rubrics, (b) examine rubric design features, (c) understand rubric use, and (d) evaluate rubric utility. A rigorous review process, involving database searching, citation tracking, and targeted review of core literature, identified 80 unique rubrics comprising a total of 265 (sub-)scales. A comprehensive analysis revealed that: (a) among 11 potential sources informing rubric development, test-external sources (e.g., literature review) were primarily used, whereas test-internal sources (e.g., performance samples) were much less consulted; (b) assessments primarily used analytic and task-type rubrics with an average of five performance levels and four scoring criteria. Rubric descriptors generally incorporated observable indicators of interpreting quality, employing both descriptive and evaluative rubric language; (c) rubrics were used by three main types of raters—interpreting practitioners, trainers, and students—to assess primarily spoken-language interpreting; and (d) rubric-based scores demonstrated moderate-to-high reliability and validity overall, though meta-analysis identified three significant moderators, including correlation coefficient type, assessment criterion, and rubric length. These findings are expected to provide guidance for assessment practice and research in interpreting.