Validating newspaper forecasts: Selective verification and the economic consequences of measurement choices

Economic forecasts extracted from newspapers can expand the historical record of expectations, but their evaluation depends on both source interpretation and outcome definition. We audit annual ProQuest exports for 1900–2010, containing 274,070 retained candidates and 175,855 deterministically scorable claims. Two human reviewers assess the same 250 historical candidates. An additional verifi- cation pass increases candidate precision from 70.8% to 77.6% against the first reviewer and from 68.4% to 76.6% against the second, while removing 11 and 7 human-accepted forecasts, respectively. Agreement on horizon and geographic scope does not consistently improve because surviving annotations are unchanged. An archived whole-article comparison further illustrates why accepted-candidate retention cannot substitute for source-level recall. On the same 9,260 employment claims, changing the tolerance band raises measured accuracy from 34.2% to 57.6%, while changing the economic target reverses the baseline-relative point estimate; the latter comparisons have intervals spanning zero. Under the saved scoring contract, price and market forecasts underperform matched fixed-direction baselines, with the price deficit more decisive. The contribution is an empirical audit of an implemented pipeline, showing which conclusions survive validation and which remain conditional on measurement choices. It establishes neither a new validation algorithm nor universal newspaper forecasting performance.