Logical AI OS, Validation Layer: Effect of a Zero-Inference External Mechanical Layer on Constraint Following — Bounded Results from Synthetic Rounds and Two Blind Public Benchmarks
A zero-inference external mechanical layer reads shape constraints (length, item count, format, paragraphstructure, required/forbidden terms, person, language, character set, relative length) from a user request,checks the answer against them, and repairs violations both after generation and during generation; theimplementation of both actions is withheld. Under frozen rules, unseen items and rule-blind gold, layer v1.8raises per-prompt full compliance of a 3B model by +14.5 points (39.3→53.8%) and of a 7B model by +17.1points (51.3→68.4%) on a synthetic constraint-dense set (S3). Layer v1.11, shot blind on the rule-only subset ofthe public ComplexBench benchmark and scored with its rule checker on whole responses, gives +3.2/+5.1points at instruction level and +5.7/+7.1 at prompt level (mappable subset: +7.4/+10.1); these subset scoresare not comparable with the benchmark’s official scores. Paired tests on per-item outcomes (McNemar;bootstrap 95% CI) make the 7B gains significant on both sets (S3 p<0.001; ComplexBench prompt-levelp=0.008) and the 3B gain significant on S3 (p=0.001) but only borderline on ComplexBench (prompt-levelp=0.08; instruction-level p=0.33). On real Chinese traffic (WildChat) the effect is near zero because explicitlyconstrained requests are rare there; the LMSYS-Chat-1M Chinese subset yielded 2 usable items and was notbuilt into a round. The first public shot (Multi-IF zh) was slightly negative (−1.8/−1.3; not distinguishable fromnoise, p=0.10/0.21) and is diagnosed as reading coverage: the extractor matched 34 of 362 mappableconstraints (type and parameter), fixed in-sample to 320/362 and out-of-sample to 145/252 on ComplexBench.Claims are bounded: feasibility is established; effectiveness is established within the shape layer; breadth isnot. Control-layer artifacts (all action implementations, rule tables and thresholds) are withheld by design; thisreport publishes effects, protocol, coverage, a failure taxonomy and boundaries.Control-layer artifacts are withheld by design (see Sections 2 and 12).
Authors
- Joe Yuan
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-10-31
- DOI
- https://doi.org/10.5281/zenodo.23826977
- Primary Topic
- Natural Language Processing Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00