Validity of interpretation and use of knowledge tests in a learning study

Purpose This article addresses issues regarding the consistency and fragility of assessment instruments in learning study based in a case. This includes what constitutes an adequate validation discussion as well as a reasonable, reliable and valid description of the research results, including necessary limitations. Design/methodology/approach Drawing on two learning studies, with negative numbers and the constitution of matter as objects of learning, the process of test development and validation is focused. The qualitative analysis of interviews and quantitative analysis of pilot test taking is discussed in relation to the validity of knowledge tests in a learning study context. Findings This paper demonstrates the process of validating the interpretation and use of knowledge test in relation to the aim of the learning study in a specific case. Findings indicate that validation of knowledge tests can be achieved and articulate what may be appropriate measures within existing limitations. Practical implications This article is an example of hands-on evaluation of validity of interpretation and use of knowledge test, suggesting methods that can be implemented on the scale of learning studies to achieve this. Originality/value Few studies have thoroughly addressed issues of validity of interpretation and use of knowledge tests in learning and lesson studies. This study does so by discussing the dominant role of knowledge tests in some studies, the current position of validity discussions in published studies and describes the process of test development and validation in a specific learning study project.