Fathom v29 / styxx v7.26.0: Gold Anchors License Nothing -- label-free auditing of LLM judge panels, an instrument that refuses, and the measured failure of sanity checks

Gold Anchors License Nothing. LLM judge panels are the de facto measurement apparatus of modern AI evaluation, and the de facto validation of those judges is a set of blatant sanity checks (exact-duplicate pairs, direct negations, honeypots). We built a label-free auditor for judge panels with refusal semantics and measured operating characteristics, sealed its datasheet on simulation (nine of nine calibration gates), and ran the measurement the practice assumes away: across four task families on a real correlated panel, blatant gold checks license label-free coverage of 0/15 while the panel is flawless on the gold items, and the failure is silent when the violation is smooth. Ladder anchors drawn from the same generator repair it by pricing (13/13) or refusing (14/15 VOID). Two extensions: on a real heterogeneous panel (Gemini Flash-Lite + local Qwen) grading TruthfulQA, gold-anchor validity is prevalence-dependent (it misses the truth by up to 0.249 as the failure becomes common, while ladder anchors cover at every prevalence); and the impossibility is made quantitative (two unlabeled-identical panels are separated by a single known-negative at a 150.8x likelihood ratio, and caught with >90% power by ~30). The method can only VOID an eval, never bless one. Instrument: styxx.anchors (styxx 7.26.0, PyPI). Every number is receipt-bound and machine-verified (OATH 88/0).

Authors

Publication Details

Journal
Open MIND
Published
2026-07-24
DOI
https://doi.org/10.5281/zenodo.21520429
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Fathom v29 / styxx v7.26.0: Gold Anchors License Nothing -- label-free auditing of LLM judge panels, an instrument that refuses, and the measured failure of sanity checks

Alexander Rodabaugh
Open MIND
Explainable Artificial Intelligence (XAI)
preprint

Fathom v29 / styxx v7.26.0: Gold Anchors License Nothing -- label-free auditing of LLM judge panels, an instrument that refuses, and the measured failure of sanity checks

Alexander Rodabaugh
preprint en

Abstract

Gold Anchors License Nothing. LLM judge panels are the de facto measurement apparatus of modern AI evaluation, and the de facto validation of those judges is a set of blatant sanity checks (exact-duplicate pairs, direct negations, honeypots). We built a label-free auditor for judge panels with refusal semantics and measured operating characteristics, sealed its datasheet on simulation (nine of nine calibration gates), and ran the measurement the practice assumes away: across four task families on a real correlated panel, blatant gold checks license label-free coverage of 0/15 while the panel is flawless on the gold items, and the failure is silent when the violation is smooth. Ladder anchors drawn from the same generator repair it by pricing (13/13) or refusing (14/15 VOID). Two extensions: on a real heterogeneous panel (Gemini Flash-Lite + local Qwen) grading TruthfulQA, gold-anchor validity is prevalence-dependent (it misses the truth by up to 0.249 as the failure becomes common, while ladder anchors cover at every prevalence); and the impossibility is made quantitative (two unlabeled-identical panels are separated by a single known-negative at a 150.8x likelihood ratio, and caught with >90% power by ~30). The method can only VOID an eval, never bless one. Instrument: styxx.anchors (styxx 7.26.0, PyPI). Every number is receipt-bound and machine-verified (OATH 88/0).

Open MIND
Peace, Justice and strong institutions
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.