Different Lenses, Same Picture? Traditional Versus Retrospective Surveys in Early Childhood Teacher Program Evaluation

As professional development (PD) programs aimed at improving early childhood educators’ teaching efficacy in integrated STEM (science, technology, engineering, mathematics) become more prevalent, understanding how best to evaluate their outcomes is increasingly important. This study compared two self-report survey designs commonly adopted in program evaluation—traditional pre-test–post-test (TPP) and retrospective pre-test (RPP)–post-test—within a year-long integrated STEM PD initiative for early childhood educators. Twenty-five educators completed the TPP survey, fifty-five completed the RPP survey, and a subset of twenty-four completed both. This study investigates whether these two designs differ in detecting change in teaching efficacy in literacy, mathematics, science, and nutrition. Findings revealed statistically significant increases across all content areas using both survey methods, with large effect sizes. Comparisons between traditional and retrospective pre-test scores showed no statistically significant differences, suggesting that response shift bias may not have meaningfully affected results in this context. The findings indicate that either survey approach can be appropriate for measuring affective outcomes such as self-efficacy. Informed by these findings and prior research, this study concludes that the choice between traditional and retrospective survey designs should be guided by practical considerations, such as program setting, time and efficiency, and constructs measured, and not only by the validity and reliability of the specific survey design. These results offer valuable guidance for evaluators seeking efficient and valid tools for assessing PD in early childhood teacher education and suggest the need for further research exploring different self-report measures in diverse educational contexts.