Reconstruction Buys Variance, Not Independence: A Tokenizer Keeps the Microstructure That Duplicates OHLCV and Drops the Signed Channels

Two-stage models for financial time series fit a tokenizer under a reconstruction objective and then an autoregressive model over the resulting codes. This assumes that a representation good enough to reconstruct from is good enough to forecast from, and it fails in a predictable direction. Reconstruction allocates code capacity by variance and covariance and never by downstream value, so a low-variance channel weakly covariant with the channels already coded is excluded deterministically rather than by chance. That is the statistical profile of trade-flow microstructure against price. On an entropy-calibrated fixture, a planted signal of known information content is lost to tokenization while the same quantity planted directly in token space is recovered by the identical backbone, localizing the loss to the tokenizer–backbone interface, not to model capacity. Re-specifying that interface restores per-bar legibility at unchanged bits per token, and we convert that property into a pre-registered hard stop on real data. The gate fired, and it named the channels. Almost all of the shortfall falls on the two signed channels, trade-flow imbalance and signed count imbalance, while the magnitude channels, which co-vary with volume, essentially clear it. The tokenizer preserves the microstructure that duplicates what OHLCV already carries and loses the microstructure that is independent of it, which is the part a microstructure model would want. That split was not anticipated, and it is the paper's result. The firing was pre-registered as the mechanism's own modal prediction, and it was obeyed: the five-cell trading ablation it gates was never run, and this paper reports no economic outcome. The finding is bounded by one budget point, one objective and one architecture: it establishes that this tokenizer does not carry the signed channels, not that none can.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-21
DOI
https://doi.org/10.5281/zenodo.22045570
Primary Topic
Complex Systems and Time Series Analysis
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Reconstruction Buys Variance, Not Independence: A Tokenizer Keeps the Microstructure That Duplicates OHLCV and Drops the Signed Channels

lakshay bhati
Zenodo (CERN European Organization for Nuclear Research)
Complex Systems and Time Series Analysis
preprint

Reconstruction Buys Variance, Not Independence: A Tokenizer Keeps the Microstructure That Duplicates OHLCV and Drops the Signed Channels

lakshay bhati
preprint en

Abstract

Two-stage models for financial time series fit a tokenizer under a reconstruction objective and then an autoregressive model over the resulting codes. This assumes that a representation good enough to reconstruct from is good enough to forecast from, and it fails in a predictable direction. Reconstruction allocates code capacity by variance and covariance and never by downstream value, so a low-variance channel weakly covariant with the channels already coded is excluded deterministically rather than by chance. That is the statistical profile of trade-flow microstructure against price. On an entropy-calibrated fixture, a planted signal of known information content is lost to tokenization while the same quantity planted directly in token space is recovered by the identical backbone, localizing the loss to the tokenizer–backbone interface, not to model capacity. Re-specifying that interface restores per-bar legibility at unchanged bits per token, and we convert that property into a pre-registered hard stop on real data. The gate fired, and it named the channels. Almost all of the shortfall falls on the two signed channels, trade-flow imbalance and signed count imbalance, while the magnitude channels, which co-vary with volume, essentially clear it. The tokenizer preserves the microstructure that duplicates what OHLCV already carries and loses the microstructure that is independent of it, which is the part a microstructure model would want. That split was not anticipated, and it is the paper's result. The firing was pre-registered as the mechanism's own modal prediction, and it was obeyed: the five-cell trading ablation it gates was never run, and this paper reports no economic outcome. The finding is bounded by one budget point, one objective and one architecture: it establishes that this tokenizer does not carry the signed channels, not that none can.

Zenodo (CERN European Organization for Nuclear Research)
Complex Systems and Time Series Analysis
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.