Schema Conformance Does Not Guarantee Decision Stability: When the Decision Is Typed but the Interpretation Isn't
Structured outputs constrain an AI system to return a valid category, schema, and numeric scores that downstream software can process, but schema adherence is not the same property as decision stability. This paper reports a controlled supplier-payment classification pilot across three frontier model configurations, using 41 texts and 287 cleaned observations. The corpus comprises one base case, 20 meaning-preserving reformulations, and 20 loaded perturbations. The results distinguish structural validity from reformulation invariance, execution repeatability, and sensitivity to material changes. One tested configuration changed its majority decision on 5 of 20 equivalent reformulations; the three configurations disagreed on 12 of 41 texts; and repeated identical inputs produced a 21.1% pairwise decision-discordance rate in one configuration. The study is a single-domain pilot and does not establish provider-only causal effects, probabilistic calibration, or universal properties of structured-output systems. It concludes that schema conformance and decision stability should be evaluated separately and complemented by deterministic validation and authoritative reference retrieval where operational decisions depend on external facts.
Authors
- José López López
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-09-29
- DOI
- https://doi.org/10.5281/zenodo.23024634
- Primary Topic
- Ethics and Social Impacts of AI
- Type
- preprint