The Missing Comparator: A Perceptual-Control Loop That Holds Language-Model Claims to an Independent Record — and the Pre-Registered Test That Could Prove It Wrong

A language model that answers from an organisation's own documents states supported and unsupported claims in the same confident voice, and nothing in the answer, or usually in the system, tells them apart. An earlier audit traced the same structure across thirty systems: an indicator held in range while the thing it stood for failed, because nothing compared the report with an independent channel. This paper builds that missing comparator into a language-model pipeline and specifies how it will be tested. The system is a runtime control loop around a hosted model, with no retraining. Before an answer is delivered, every factual claim is checked against an independent, read-only record. Two quantities are held to explicit references: - agreement of the answer's claims with the record; - coverage of the question by supported content. A higher level sets the lower references and can never lower the agreement target. Claims the loop cannot support are rewritten, removed, marked unchecked or shown as conflicts; they are never delivered as plain assertions. A hash-chained, per-claim ledger records what was checked, against what, and how. The design controls agreement with records, not truth, and the test is built to show that limit. Evaluation uses the Test for the Controlled Variable from perceptual control theory. The controlled variables are disturbed with their references held fixed. A clamped-feedback test asks whether corrections depend on the loop's own measurements. The loop is compared with verification built from the same components at matched cost. Eight hypotheses are pre-registered, each with the result that would count against it. This version reports no results. Version 2 will report them under this record, whatever they show.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-17
DOI
https://doi.org/10.5281/zenodo.23428833
Primary Topic
Ethics and Social Impacts of AI
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

The Missing Comparator: A Perceptual-Control Loop That Holds Language-Model Claims to an Independent Record — and the Pre-Registered Test That Could Prove It Wrong

Łukasz Diener
Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
preprint

The Missing Comparator: A Perceptual-Control Loop That Holds Language-Model Claims to an Independent Record — and the Pre-Registered Test That Could Prove It Wrong

Łukasz Diener
preprint en

Abstract

A language model that answers from an organisation's own documents states supported and unsupported claims in the same confident voice, and nothing in the answer, or usually in the system, tells them apart. An earlier audit traced the same structure across thirty systems: an indicator held in range while the thing it stood for failed, because nothing compared the report with an independent channel. This paper builds that missing comparator into a language-model pipeline and specifies how it will be tested. The system is a runtime control loop around a hosted model, with no retraining. Before an answer is delivered, every factual claim is checked against an independent, read-only record. Two quantities are held to explicit references: - agreement of the answer's claims with the record; - coverage of the question by supported content. A higher level sets the lower references and can never lower the agreement target. Claims the loop cannot support are rewritten, removed, marked unchecked or shown as conflicts; they are never delivered as plain assertions. A hash-chained, per-claim ledger records what was checked, against what, and how. The design controls agreement with records, not truth, and the test is built to show that limit. Evaluation uses the Test for the Controlled Variable from perceptual control theory. The controlled variables are disturbed with their references held fixed. A clamped-feedback test asks whether corrections depend on the loop's own measurements. The loop is compared with verification built from the same components at matched cost. Eight hypotheses are pre-registered, each with the result that would count against it. This version reports no results. Version 2 will report them under this record, whatever they show.

Zenodo (CERN European Organization for Nuclear Research)
Ethics and Social Impacts of AI
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.