Linkage Is Not Validity: Auditing the Actual Basis of a Belief
Linking a reason report to the right episode does not establish that it measures the reason actually used. Asking for a reason can prompt reconsideration, so a correctly linked report may describe a different inference. This paper develops a method for auditing a past basis. It models the probe separately from the original inference and shows when linked fallible indicators can be calibrated without a gold standard. Worked examples show how residual dependence and calibration drift undermine point identification and what can still be learned from bounded error or from the recorded overlap alone. A finite design criterion tells auditors which records must be available together for a given score; even all pairwise records can miss a three-way defect. An application to language-model faithfulness distinguishes an answer switch from actual reliance on a hint. The pump examples use synthetic data, and the simulator study is a proposed design rather than an empirical result.
Authors
- Lorand Bruhacs (ORCID: https://orcid.org/0009-0004-6751-0715)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-15
- DOI
- https://doi.org/10.5281/zenodo.24248948
- Primary Topic
- Explainable Artificial Intelligence (XAI)
- Type
- preprint