Variation theory meets diagnostic classification models: the case of assessment in a learning study on subtraction with negative numbers

Purpose The study reports a learning study on subtraction with negative numbers based on Variation Theory (VT), with special focus on exploring the conditions for using diagnostic classification models (DCMs) for assessment. DCMs offer a multidimensional statistical modeling approach where the dimensions may mimic CAs (CAs) in VT. The study aims to within a learning study develop a knowledge test that meets DCM standards and to assess its usefulness in that context. Design/methodology/approach Data comes from a learning study based on VT, in grade 6 in three Swedish schools. The developed teaching relied on four identified CAs that were also used to develop a knowledge test meeting the standards for DCM. The methodology involved iterative cycles of test development and statistical model evaluation, alongside qualitative methods such as think-aloud interviews. Findings The developed test meets statistical standards implied by DCMs. Challenges in operationalizing CAs in test items are identified. Detailed information about how test-takers discerned CAs becomes available, which also may be used for the development of teaching and the articulation of CAs. In particular, the DCM analysis provided individual profiles regarding CAs, supporting a nuanced understanding of learning outcomes at the group level. Originality/value The bridging VT and DCMs offer a comprehensive assessment of learning outcomes, emphasizing the importance of aligning tests with the object of learning. The findings suggest that DCMs can complement qualitative methods, providing a systematic and detailed analysis of students' knowledge.