What Does "Not Yet Taught" Mean? Decomposing Teacher-Reported Opportunity to Learn and Item-Level Achievement in TIMSS 2023

Background Secondary analyses of international assessments usually treat teachers’ reports that a topic has “not yet” been taught as a lack of opportunities to learn (OTL). The Trends in International Mathematics and Science Study (TIMSS) 2023 Insights Report found near-zero correlations between teacher-reported coverage and achievement and called for country-level examination. Interpreting such correlations requires establishing what “not yet taught” responses represent and at what level achievement should be measured. I illustrate this procedure using Japanese Grade 8 data. Methods In this cross-sectional secondary analysis, topics with coverage below 95% were classified by cross-referencing Test-Curriculum Matching Analysis (TCMA) judgments, national grade placement, and textbook scheduling. For topics consistent with within-year scheduling, I estimated differences in item percent correct between students whose teachers reported the topic as taught or not yet taught, computed standard errors within jackknife replicates, and fitted multilevel and cross-domain contrast models. Results The seven low-coverage topics fell into four candidate types: only probability (coverage 77.2%) was fully included in the TCMA, placed in Grade 8, and scheduled near the March assessment, consistent with instructional timing rather than omission. Across 12 probability items, the taught group scored 1.1 points higher (design-based 95% CI −5.2 to +7.4); supporting models yielded similar results (multilevel odds ratio 1.07, 0.84–1.37). Combining item-level standard errors assuming independence would have halved the standard error (1.64 vs. 3.22), because group status was essentially constant within schools (27 not-yet-taught schools). Conclusions The interval excludes declines larger than approximately 7.4 points but not smaller ones; the association is observational. “Not yet taught” responses must be interpreted in light of questionnaire content, national scheduling, and assessment timing; achievement must be measured where coverage varies; and uncertainty must be aggregated while accounting for dependence among items. The instructional sensitivity of these items remains untested.