A Loose Ceiling, Not a Certificate: Auditing Fisher-Separability Bounds on Hallucination Detectability Across Architectures

Status: Manuscript (preprint). Can the ceiling on hallucination detectability be read off a representation before any probe is trained? Fisher separability J yields a closed-form, training-free value that empirically upper-bounds linear-probe AUROC via the binormal identity Φ(√(J/2)). Rerunning six models (GPT-2 117M/Medium, Qwen2.5 0.5B–7B) under one matched real-generation pipeline, we find the bound is never violated but badly loose: it over-predicts probe AUROC by 0.32–0.41 under exact-match/F1 labels (and 0.18–0.30 under an independent semantic judge, at re-selected layers). The central question is how much of that gap is the estimator rather than the representation, and the answer is: most of it. Under a label-permutation null on the same activations – no class signal by construction – the estimator still returns a "certificate" of 0.87–0.94 for GPT-2 117M, GPT-2 Medium and Qwen2.5-0.5B, which against their same-layer probe AUROCs of 0.55–0.56 is 86–92% of the gap we report for those three models. Observed J exceeds its null at p < 0.001–0.045 uncorrected for four of six models, and is not distinguishable from random labels at all for the other two (p=0.195, 0.380). Holding all 38 significance tests in the paper to one Holm-Bonferroni family leaves 3 surviving. We also retract an invariance claim from an earlier version of this work: an apparent rotation-sensitivity of Ĵ, which we had read as a basis-dependence of fixed-rank truncation, traces entirely to an unseeded randomized SVD and vanishes under a deterministic solver. We report the null control and the determinism check as the two diagnostics any Fisher-style detectability ceiling should carry – offered here because they overturned our own headline rather than confirmed it. Code and result files: doi:10.5281/zenodo.23625500 (GitHub: geom-proof).