Rethinking TIMSS quality assurance: utilizing neural network models with regression-based bias mitigation strategies for validating country-level math and science achievement scores

Abstract This study presents a novel approach to improving quality assurance in the Trends in International Mathematics and Science Study (TIMSS). It utilizes a combination of neural networks and regression analysis to validate country-level achievement scores. Concerns over the integrity of TIMSS test scores necessitate robust quality control methods. The research described here developed and piloted new statistical indicators to identify countries with apparently unusual results, aiming to enhance the efficacy of the TIMSS quality assurance process. The methodology leverages the unique design of TIMSS, which assesses 4th and 8th graders every 4 years, to generate model-based predictions for country-level average scores in math and science. The comparison of these predictions against actual results supported the feasibility of this approach. This study also investigated the correlation between large positive residuals and substantial changes in sample quality or population definition. While no strong correlations were found, this investigation contributes valuable insights for future TIMSS assessments.