Pre-registration of a replication of the LTSF-Linear versus Transformer comparison, with a matched tuning budget (arXiv 2205.13504)
Frozen pre-registration (3 Nov 2026) of a private replication of the claim of Zeng et al. (arXiv 2205.13504; code cure-lab/LTSF-Linear, commit 0c113668) that LTSF-Linear models outperform the Transformer forecasters FEDformer, Autoformer and Informer in all cases and often by 20-50 percent. Scope: ETTh1 and ETTh2 at horizons 96, 192, 336, 720 (8 cells). Two verdict lines: arm R (the authors' released scripts, compared with the printed Transformer numbers and with an independent Transformer reproduction control) and arm P (the same number of tuning configurations, K = 8, for each of six models, look-back included, selection by validation MSE). Pre-set bounds (median improvement interval lower end above 6.8 percent, half the printed median over these cells; at most 1 of 8 cells wrong-side), controls that can fire (Repeat baseline reproduces the printed numbers within 1 percent, zero-predictor, reproduction and budget-parity controls). Contains the decision code with its planted-effect table, job design, harness, extraction of the paper's Table 2, cost estimate and the run kernel. Deposited before any confirmatory run. Outcome wording: Replicated / Did not replicate / Inconclusive under protocol X.
Authors
- John Goodman (ORCID: https://orcid.org/0009-0008-4080-8043)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-03
- DOI
- https://doi.org/10.5281/zenodo.23907284
- Primary Topic
- Time Series Analysis and Forecasting
- Type
- article
- Field-Weighted Citation Impact
- 0.00