Pre-registration of a replication of the LTSF-Linear versus Transformer comparison, with a matched tuning budget (arXiv 2205.13504)

Frozen pre-registration (3 Nov 2026) of a private replication of the claim of Zeng et al. (arXiv 2205.13504; code cure-lab/LTSF-Linear, commit 0c113668) that LTSF-Linear models outperform the Transformer forecasters FEDformer, Autoformer and Informer in all cases and often by 20-50 percent. Scope: ETTh1 and ETTh2 at horizons 96, 192, 336, 720 (8 cells). Two verdict lines: arm R (the authors' released scripts, compared with the printed Transformer numbers and with an independent Transformer reproduction control) and arm P (the same number of tuning configurations, K = 8, for each of six models, look-back included, selection by validation MSE). Pre-set bounds (median improvement interval lower end above 6.8 percent, half the printed median over these cells; at most 1 of 8 cells wrong-side), controls that can fire (Repeat baseline reproduces the printed numbers within 1 percent, zero-predictor, reproduction and budget-parity controls). Contains the decision code with its planted-effect table, job design, harness, extraction of the paper's Table 2, cost estimate and the run kernel. Deposited before any confirmatory run. Outcome wording: Replicated / Did not replicate / Inconclusive under protocol X.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-03
DOI
https://doi.org/10.5281/zenodo.23907284
Primary Topic
Time Series Analysis and Forecasting
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Pre-registration of a replication of the LTSF-Linear versus Transformer comparison, with a matched tuning budget (arXiv 2205.13504)

John Goodman
Zenodo (CERN European Organization for Nuclear Research)
Time Series Analysis and Forecasting
article

Pre-registration of a replication of the LTSF-Linear versus Transformer comparison, with a matched tuning budget (arXiv 2205.13504)

John Goodman
article en

Abstract

Frozen pre-registration (3 Nov 2026) of a private replication of the claim of Zeng et al. (arXiv 2205.13504; code cure-lab/LTSF-Linear, commit 0c113668) that LTSF-Linear models outperform the Transformer forecasters FEDformer, Autoformer and Informer in all cases and often by 20-50 percent. Scope: ETTh1 and ETTh2 at horizons 96, 192, 336, 720 (8 cells). Two verdict lines: arm R (the authors' released scripts, compared with the printed Transformer numbers and with an independent Transformer reproduction control) and arm P (the same number of tuning configurations, K = 8, for each of six models, look-back included, selection by validation MSE). Pre-set bounds (median improvement interval lower end above 6.8 percent, half the printed median over these cells; at most 1 of 8 cells wrong-side), controls that can fire (Repeat baseline reproduces the printed numbers within 1 percent, zero-predictor, reproduction and budget-parity controls). Contains the decision code with its planted-effect table, job design, harness, extraction of the paper's Table 2, cost estimate and the run kernel. Deposited before any confirmatory run. Outcome wording: Replicated / Did not replicate / Inconclusive under protocol X.

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 16%
Time Series Analysis and Forecasting
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon