A semantic layer architecture for removing schema identifiers from external LLM prompts, with model-dependent accuracy trade-offs

Abstract The adoption of large language model (LLM)-based AI systems in credit rating agencies introduces a structural tension between operational efficiency and data security. This study examines a Semantic Layer as a technical mechanism for reducing this tension, and shows that whether it removes designated schema identifiers from the LLM-facing prompt depends on an implementation detail invisible to a schema-level analysis: whether the concept-to-SQL mapping is itself withheld from the LLM. An initial implementation (v1) includes this mapping in the prompt and removes none of the designated schema identifiers from the LLM-facing payload; a corrected implementation (v2) withholds it entirely and removes all of them, verified exhaustively over the complete payload. We scope this security claim explicitly: it concerns the removal of designated schema identifiers from the prompt, not a general reduction in sensitive-information exposure, and we specify the associated threat model and its out-of-scope leakage channels. Across two benchmarks (a purpose-built credit-rating benchmark designed by the author alongside the Semantic Layer specification, and the externally-authored BIRD financial benchmark) and three LLM tiers spanning a capability range, we find that accuracy is model-dependent rather than uniformly preserved: on the author-designed credit-rating benchmark v2 improves with model capability and, at the strongest tier, is numerically highest among the three conditions (60.7% vs. 58.7% for v1 and 52.0% for Text-to-SQL), although with $$n=60$$ questions this strongest-tier gap does not reach statistical significance, whereas on the externally-authored benchmark v2 trails both baselines at every tier, with the in-benchmark-versus-external gap widening as capability increases. We characterize this pattern as selective encapsulation , presented as an empirical design lesson rather than a predictive theory. Rather than extending Parnas’s (1972) Information Hiding principle, we empirically demonstrate that it is easily violated at LLM interfaces and requires payload-level verification. We also contribute the first domain-specific NL-to-SQL benchmark for credit-rating contexts, together with an execution-based self-check of its gold queries performed by the benchmark designer.