Even the LLM Needs a Layer: a refusal battery for knowledge-graph competency evaluation
This record is a dated artifact record for a forthcoming paper, deposited before the paper exists and before the battery was run against the serving store. It argues from the published evidence that a language layer alone cannot certify its own answers. CodeSimpleQA measured the strongest frontier model at 62.9 percent on stable coding facts, with grounded retrieval near 71 percent against a parametric ceiling near 42 percent. GPTKB's 101-million-triple knowledge base built from a model showed only 31 percent of sampled triples verifiably true. KGQAGen's audit of 16 KGQA benchmarks found average factual correctness of 57 percent. The artifact is a 50-row human-authored refusal battery for knowledge-graph competency evaluation, frozen in advance: 40 knowledge rows, 8 refusal rows, 2 canary rows. The battery executes as a release gate against the live serving store, where an empty result over a complete schema is certified absence and a refusal is distinguishable from a failed retrieval. A paper describing the evaluation is under preparation. Files are restricted during the priority window as an independent-development record. Version 1, November 2026.
Authors
- Kazi Golam Rabbany (ORCID: https://orcid.org/0009-0007-4549-0815)
Institutions
- Narxoz University (KZ)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-22
- DOI
- https://doi.org/10.5281/zenodo.24430248
- Primary Topic
- Topic Modeling
- Type
- preprint