Right Number, Forged Pedigree: A Preregistered Minimal Study of False Auditability in LLM Answers over XBRL Knowledge Graphs

Preregistered analysis plan, frozen before any call to a real language model. Files: plan, configuration, frozen corpus (sha256 inside), dry-run cost record, offline gate results. Graph-grounded LLM systems are often called auditable because their answers cite graph nodes. A citation only supports an audit if it is valid: the cited nodes exist and the answer can be recomputed from them. This study measures two things on real models. Study A asks how often a model gives a numerically correct answer whose citation is not valid (false auditability). Study B asks whether a model acting as an auditor accepts a correct answer whose cited provenance has been forged. The study is deliberately small (30 firm-years, 240 items, six open-weight models) and is designed to answer one question each, with offline controls that fix what a perfect and a blind system would score.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-17
DOI
https://doi.org/10.5281/zenodo.24311557
Primary Topic
Explainable Artificial Intelligence (XAI)
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Right Number, Forged Pedigree: A Preregistered Minimal Study of False Auditability in LLM Answers over XBRL Knowledge Graphs

Oussama Chahed
2 links
Zenodo (CERN European Organization for Nuclear Research)
Explainable Artificial Intelligence (XAI)
preprint

Right Number, Forged Pedigree: A Preregistered Minimal Study of False Auditability in LLM Answers over XBRL Knowledge Graphs

Oussama Chahed
preprint en

Abstract

Preregistered analysis plan, frozen before any call to a real language model. Files: plan, configuration, frozen corpus (sha256 inside), dry-run cost record, offline gate results. Graph-grounded LLM systems are often called auditable because their answers cite graph nodes. A citation only supports an audit if it is valid: the cited nodes exist and the answer can be recomputed from them. This study measures two things on real models. Study A asks how often a model gives a numerically correct answer whose citation is not valid (false auditability). Study B asks whether a model acting as an auditor accepts a correct answer whose cited provenance has been forged. The study is deliberately small (30 firm-years, 240 items, six open-weight models) and is designed to answer one question each, with offline controls that fix what a perfect and a blind system would score.

Zenodo (CERN European Organization for Nuclear Research)
Université Paris Dauphine-PSL (FR)
Explainable Artificial Intelligence (XAI)
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon