Eight Retractions in Twenty-Four Hours: A Log of What Broke While Measuring Small Language Models on One Home PC

Over roughly twenty-four hours of continuous measurement on a single consumer GPU, eight claims were retracted. Each retraction was recorded at the moment it died, in a fixed schema that includes a field which cannot be reconstructed afterwards: why it looked true. Three of the eight were caused by the author's own tools reporting events that had not happened — a reproduction script that printed a context-overflow rejection for a request it never sent, a static checker that raised nine false findings, and an aggregator that reported zero paired cells because its stratification key included the model name. Three were framing reversals, in which changing how a task was posed reversed the result. Two were directions asserted from single-digit samples. The argument is narrow: at this scale the researcher is also the instrument-maker, and tool-generated non-events are a failure mode of the same order as statistical error, yet they are not usually reported. The log is published rather than the lessons, because the lessons are short and the log is what makes them believable. Version 1 contains cases A through H. Later versions add cases; the case identifiers are stable and will not be reassigned. The machine-readable ledger is included alongside the text.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-21
DOI
https://doi.org/10.5281/zenodo.22048923
Primary Topic
Data Analysis with R
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Eight Retractions in Twenty-Four Hours: A Log of What Broke While Measuring Small Language Models on One Home PC

satoshi akiyama
Zenodo (CERN European Organization for Nuclear Research)
Data Analysis with R
article

Eight Retractions in Twenty-Four Hours: A Log of What Broke While Measuring Small Language Models on One Home PC

satoshi akiyama
article en

Abstract

Over roughly twenty-four hours of continuous measurement on a single consumer GPU, eight claims were retracted. Each retraction was recorded at the moment it died, in a fixed schema that includes a field which cannot be reconstructed afterwards: why it looked true. Three of the eight were caused by the author's own tools reporting events that had not happened — a reproduction script that printed a context-overflow rejection for a request it never sent, a static checker that raised nine false findings, and an aggregator that reported zero paired cells because its stratification key included the model name. Three were framing reversals, in which changing how a task was posed reversed the result. Two were directions asserted from single-digit samples. The argument is narrow: at this scale the researcher is also the instrument-maker, and tool-generated non-events are a failure mode of the same order as statistical error, yet they are not usually reported. The log is published rather than the lessons, because the lessons are short and the log is what makes them believable. Version 1 contains cases A through H. Later versions add cases; the case identifiers are stable and will not be reassigned. The machine-readable ledger is included alongside the text.

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 8%
Data Analysis with R
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Eight Retractions in Twenty-Four Hours: A Log of What Broke While Measuring Small Language Models on One Home PC — satoshi akiyama · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS