Prequential Streaming Evaluation of Stateful Time-Series Forecasters with Online Conformal Calibration

Real forecasting is streaming: observations arrive one by one, the model updates, and its prediction intervals must stay calibrated as the series drifts. Yet every widely used forecasting benchmark harness evaluates models statelessly on rolling windows, carrying no state across windows and never running a test-then-train loop. This preprint addresses that verified gap with a prequential streaming evaluation harness for stateful forecasters, released as the open-source package preqts: it implements Dawid's predict-then-observe-then-update protocol behind a StatefulForecaster protocol with an explicit covariate-arrival policy, and records four trajectories per method (rolling scaled error, rolling empirical coverage vs the nominal level, interval width, and cumulative compute cost, the last of which separates a genuinely stateful constant-cost model from a replay-everything wrapper). The harness is used to test a concrete claim: that online conformal calibration keeps streaming intervals at their nominal level under distribution shift. On an atlas of fifteen series spanning distinct regimes (level shift, chaos, heteroskedasticity, intermittency, long memory, multi-seasonality, exogenous promotions, and real UCI/M4/PM2.5 data) plus white-noise and random-walk honesty controls, wrapping a Theta point forecaster in Adaptive Conformal Inference or Conformal-PID control reduces the mean absolute coverage error at the nominal 80 percent level from 0.042 (raw Theta) and 0.060 (seasonal-naive) to 0.0057 and 0.0060 respectively, a roughly seven-to-tenfold reduction, and caps the worst-case per-series coverage error at 0.022 versus 0.11-0.31 uncalibrated, without changing the point forecast. The calibration holds on the honesty controls, where a spurious skill gap would flag harness leakage. The contribution is the reusable stateful-streaming harness and a leakage-guarded, artifact-backed demonstration of where online conformal calibration earns its keep. Harness: https://pypi.org/project/preqts/ . Atlas testbed (code, committed artifacts, workbench): https://github.com/fsantibanezleal/CAOS_RES_ChronoScope (MIT), live at https://chronoscope.fasl-work.com .