Participation quality and estimator-specific bias in TALIS: comparing survey estimates with administrative benchmarks

Abstract This study examines whether commonly used indicators of survey quality in international large-scale assessments adequately capture the validity of specific survey estimates. Using data from the OECD Teaching and Learning International Survey (TALIS) 2013, 2018, and 2024, the analysis compares survey-based estimates of teacher characteristics with external administrative benchmarks from Eurostat across European education systems. Two indicators are examined: the proportion of female teachers and the proportion of teachers working part-time. Country fixed-effects models are used to assess whether participation conditions, including school participation, teacher response rates, replacement procedures, and unused sampling slots, are associated with deviations between survey estimates and population benchmarks. Results show that deviations in gender composition are generally small and only weakly related to participation quality. In contrast, estimates of part-time employment display larger and systematically negative deviations, indicating persistent underrepresentation of part-time teachers in achieved survey samples. Unexpectedly, stronger participation conditions are associated with larger discrepancies for part-time employment, suggesting that higher participation rates do not necessarily imply more accurate estimates. The findings support the argument that survey quality in international assessments is estimator specific and demonstrate the importance of validating survey estimates against external administrative benchmarks.