Coverage Is Not Evidence: How News Miscalibrates LLM Forecasters, and What Survives Correcting It

Forecasting language models are judged by accuracy, but how they use the news is rarely measured. We read the same questions with no evidence, with the real headlines and with unrelated placebo headlines, using four open models from three families (7B to 120B parameters) and three proprietary frontier models. Coverage is treated as evidence. In open models, headlines about a question raise the mean forecast by 11–22 percentage points across four question sets, including a fresh set with registered predictions, and on questions written from headlines the rise is nearly as large for events that did not happen as for those that did. The push grows with the number of headlines, persists when headlines point the other way, survives thinking mode and is not removed by warning the model; unrelated headlines do not raise forecasts. In frontier models the push is smaller (6–10 points) and mostly tied to the outcome, but its outcome-unrelated part (3–7 points) remains. Reading headlines one at a time and averaging halves the push without losing ranking skill, and subtracting the push, measured without outcomes, recovers about 70% of the attainable improvement where the question-only forecast is calibrated on average. Auditing sixteen inference-time fixes and ten further rules on held-out questions, we find that only an anchored shrink toward the base rate survives, because it corrects both errors the news creates: a YES bias on headline-built questions and over-dispersion on market questions. Benchmarks built from headlines and open models therefore partly measure a model's reaction to coverage, not its forecasting skill.This record contains the manuscript (preprint), the anonymised reproducibility package (questions, digested prompts, model replies, label audit, protocol log and analysis scripts, with standalone scripts that reproduce the main tables), and the LaTeX source.