Creating Short Forms of Early Childhood Development Measures: A Framework for Quantifying Statistical, Conceptual, and Practical Tradeoffs in Direct Assessment

• Short forms of direct assessments of ECD can generate rich data on ECD at lower cost and complexity. • When creating short forms, the statistical, conceptual, and practical properties of items are important to consider to create more valid and easier to use assessments. • A balanced short-form IDELA consists of 8 subtasks and generates adequate information on children’s overall early learning and development. Direct assessments of early childhood development (ECD) are a cornerstone of research in developmental psychology and are increasingly used to evaluate programs and policies in lower- and middle-income countries. Despite strong psychometric properties, these assessments are too expensive and time consuming for use in large-scale monitoring or national-level surveys. Short forms of direct assessments can provide some benefits of direct assessment at substantially lower cost and complexity. However, selecting the best items for inclusion on shorter forms is not a straightforward task. Traditional approaches to creating short forms, which rely on statistical properties of items, can neglect important non-statistical considerations and result in narrowed construct coverage that does not maximize improvements in usability. Automated Test Assembly is an ideal approach to generate optimal forms given numerical constrains, but can be difficult to operationalize. This paper proposes a theoretical framework for an empirically driven, human-centered process to create short forms of ECD direct assessments. It builds on the goals of Automated Test Assembly in an accessible manner by evaluating items across three dimensions: statistical (how reliably items distinguish between children with higher and lower development), conceptual (how representative items are of the constructs being assessed), and practical (how time-consuming and difficult items are to administer). Having defined this framework, the paper then applies it to the International Development and Early Learning Assessment, a popular direct assessment of ECD, to suggest a general-purpose short form selected after considering these three dimensions.