Analysis of process data to advance digital-based multilingual assessments

Unlike traditional large-scale assessments, digital-based assessments collect process data that provide information about test-takers' actions in the assessment platform while solving test items, e.g., response and reaction times, number of clicks, and skipping. In multilingual digital-based assessments, the comparability of test scores across different assessment languages is essential and challenges assessment design principles regarding fairness. To achieve comparability, test items must be independent of language specificities, test-takers should be homogeneous in their ability to solve the test items, and the test setting must be standardised across test-takers. Using process data in assessments is often limited to identifying cognitive response mechanisms. On the other hand, little is known about how process data can contribute to multilingual assessments' test development and administration process, hence, quality assurance. Based on the existing literature, we conceptualise a framework with five aspects of how process data can be used for assessment quality improvement. In addition, we set these aspects into the context of multilingual digital-based assessments. We illustrate three ways in which process data can contribute to the quality assurance of standardised educational digital-based assessments - a) assisting in identifying suspicious items or blocks across different languages, b) investigating potential issues of position effects and how they might affect different languages and c) clustering test-takers to understand their evolving behaviour. This strand of research aims to contribute to a better understanding of how process data in digital-based assessments can contribute to quality assurance and inform assessment developers, practitioners, and researchers in large-scale assessments.