Real-life applications of competence-based test development to the construction, improvement, and shortening of tests

Abstract Competence-based test development (CbTD) is a novel method for constructing tests that are as informative as possible about the competence state (the set of skills an individual masters) underlying the item responses. If desired, the tests can also be minimal, meaning that no item can be eliminated without reducing their informativeness. To demonstrate the usefulness and usability of CbTD in practice, this work illustrates real-life applications to the construction of a test from scratch, and the improvement and shortening of existing tests. In particular, a test for the assessment of students’ mathematical formulation is constructed from scratch and validated using empirical data collected from middle and high school students. A booklet of the TIMSS 2003 eighth grade mathematics test is improved by adding a minimal set of items appropriately selected from another booklet of the same test, and the resulting booklet is compared with the original one. Finally, a shortened form of a well-known fraction subtraction test is developed and compared to other forms of the same length. The findings suggest that CbTD can help to improve the development of tests by identifying specific items needed to assess particular competence states. It can also support assessment practice in at least two respects: by enabling an unambiguous and precise identification of individuals’ competence states, which helps plan personalized interventions, and by allowing for assessments that are shorter but just as accurate as longer ones.