Separating prior knowledge from acquired knowledge: An individual differences analysis of PISA - learning in the digital world

The innovative PISA domain “Learning in the digital world (LDW)” integrates the assessment of knowledge and skills with opportunities for learning. To investigate whether learning took place during the assessment, we analyzed data from 737 German PISA 2022 students and modeled individual differences in test performance that could not be explained by students'prior knowledge and skills. We then related these differences to learning activity (the use of worked examples during a task) and learning prerequisites (general intelligence and mastery orientation) to validate their interpretation as performance based on knowledge and skills acquired during the task. We found substantial remaining variance in performance after controlling for prior knowledge (21 % of variance). Significant relationships of these differences with both learning prerequisites and learning activity provide further evidence for interpreting found differences as a result of learning during the task. This study provides initial evidence that a learning process takes place during PISA-LDW assessments. Students that made use of the provided learning opportunities and/or have high intelligence consistently performed better than would be expected based on their prior knowledge. The use of learning opportunities was related to the learning goal orientation of students. Digital environments should be designed in a way that is conducive to learning goal orientations and provide explicit learning opportunitites. • Acquired knowledge can be approximated by residuals of performance. • The interpretation of these residuals is supported by validity arguments. • Students who used worked examples outperformed their prior knowledge. • High intelligence students outperformed their prior knowledge. • Learning goal orientation predicted usage of worked examples.