Right Number, Forged Pedigree: A Preregistered Minimal Study of False Auditability in LLM Answers over XBRL Knowledge Graphs
Preregistered analysis plan, frozen before any call to a real language model. Files: plan, configuration, frozen corpus (sha256 inside), dry-run cost record, offline gate results. Graph-grounded LLM systems are often called auditable because their answers cite graph nodes. A citation only supports an audit if it is valid: the cited nodes exist and the answer can be recomputed from them. This study measures two things on real models. Study A asks how often a model gives a numerically correct answer whose citation is not valid (false auditability). Study B asks whether a model acting as an auditor accepts a correct answer whose cited provenance has been forged. The study is deliberately small (30 firm-years, 240 items, six open-weight models) and is designed to answer one question each, with offline controls that fix what a perfect and a blind system would score.
Authors
- Oussama Chahed (ORCID: https://orcid.org/0009-0009-8062-4377)
Institutions
- Université Paris Dauphine-PSL (FR)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-17
- DOI
- https://doi.org/10.5281/zenodo.24311557
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint