Broken Input Looks Exactly Like Bad Reasoning

When a language model pipeline produces a wrong judgment, the field's default diagnosis is that the model reasoned badly. This paper documents four cases from a production legal citation verification system where that diagnosis would have been wrong. In each, the model reasoned correctly and consistently over text it had never been shown in complete or accurate form. A text-handling defect upstream of the model produced the wrong output — a hyphen, a character cap, a sentence boundary, and a privacy filter. The distinction matters because the two failure modes require opposite fixes and are observationally identical from the output side. The fourth case is a distinct and more important sub-class: the corruption was not a defect at all. A privacy-redaction pass working exactly as designed destroyed the evidence it existed to make safe to analyse, and the damage propagated into a second subsystem where it silently disabled a human correction. All four are named against public opinions, with the specific text, so the claims can be checked.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-24
DOI
https://doi.org/10.5281/zenodo.22072931
Primary Topic
Topic Modeling
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Broken Input Looks Exactly Like Bad Reasoning

Benjamin Schulz
Zenodo (CERN European Organization for Nuclear Research)
Topic Modeling
preprint

Broken Input Looks Exactly Like Bad Reasoning

Benjamin Schulz
preprint en

Abstract

When a language model pipeline produces a wrong judgment, the field's default diagnosis is that the model reasoned badly. This paper documents four cases from a production legal citation verification system where that diagnosis would have been wrong. In each, the model reasoned correctly and consistently over text it had never been shown in complete or accurate form. A text-handling defect upstream of the model produced the wrong output — a hyphen, a character cap, a sentence boundary, and a privacy filter. The distinction matters because the two failure modes require opposite fixes and are observationally identical from the output side. The fourth case is a distinct and more important sub-class: the corruption was not a defect at all. A privacy-redaction pass working exactly as designed destroyed the evidence it existed to make safe to analyse, and the damage propagated into a second subsystem where it silently disabled a human correction. All four are named against public opinions, with the specific text, so the claims can be checked.

Zenodo (CERN European Organization for Nuclear Research)
Computer Algorithms for Medicine (AT)
Peace, Justice and strong institutions
Topic Modeling
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Broken Input Looks Exactly Like Bad Reasoning — Benjamin Schulz · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS