Logical AI OS, Validation Layer: Effect of a Zero-Inference External Mechanical Layer on Constraint Following — Bounded Results from Synthetic Rounds and Two Blind Public Benchmarks

A zero-inference external mechanical layer reads shape constraints (length, item count, format, paragraphstructure, required/forbidden terms, person, language, character set, relative length) from a user request,checks the answer against them, and repairs violations both after generation and during generation; theimplementation of both actions is withheld. Under frozen rules, unseen items and rule-blind gold, layer v1.8raises per-prompt full compliance of a 3B model by +14.5 points (39.3→53.8%) and of a 7B model by +17.1points (51.3→68.4%) on a synthetic constraint-dense set (S3). Layer v1.11, shot blind on the rule-only subset ofthe public ComplexBench benchmark and scored with its rule checker on whole responses, gives +3.2/+5.1points at instruction level and +5.7/+7.1 at prompt level (mappable subset: +7.4/+10.1); these subset scoresare not comparable with the benchmark’s official scores. Paired tests on per-item outcomes (McNemar;bootstrap 95% CI) make the 7B gains significant on both sets (S3 p<0.001; ComplexBench prompt-levelp=0.008) and the 3B gain significant on S3 (p=0.001) but only borderline on ComplexBench (prompt-levelp=0.08; instruction-level p=0.33). On real Chinese traffic (WildChat) the effect is near zero because explicitlyconstrained requests are rare there; the LMSYS-Chat-1M Chinese subset yielded 2 usable items and was notbuilt into a round. The first public shot (Multi-IF zh) was slightly negative (−1.8/−1.3; not distinguishable fromnoise, p=0.10/0.21) and is diagnosed as reading coverage: the extractor matched 34 of 362 mappableconstraints (type and parameter), fixed in-sample to 320/362 and out-of-sample to 145/252 on ComplexBench.Claims are bounded: feasibility is established; effectiveness is established within the shape layer; breadth isnot. Control-layer artifacts (all action implementations, rule tables and thresholds) are withheld by design; thisreport publishes effects, protocol, coverage, a failure taxonomy and boundaries.Control-layer artifacts are withheld by design (see Sections 2 and 12).

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-31
DOI
https://doi.org/10.5281/zenodo.23826977
Primary Topic
Natural Language Processing Techniques
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Logical AI OS, Validation Layer: Effect of a Zero-Inference External Mechanical Layer on Constraint Following — Bounded Results from Synthetic Rounds and Two Blind Public Benchmarks

Joe Yuan
Zenodo (CERN European Organization for Nuclear Research)
Natural Language Processing Techniques
article

Logical AI OS, Validation Layer: Effect of a Zero-Inference External Mechanical Layer on Constraint Following — Bounded Results from Synthetic Rounds and Two Blind Public Benchmarks

Joe Yuan
article en

Abstract

A zero-inference external mechanical layer reads shape constraints (length, item count, format, paragraphstructure, required/forbidden terms, person, language, character set, relative length) from a user request,checks the answer against them, and repairs violations both after generation and during generation; theimplementation of both actions is withheld. Under frozen rules, unseen items and rule-blind gold, layer v1.8raises per-prompt full compliance of a 3B model by +14.5 points (39.3→53.8%) and of a 7B model by +17.1points (51.3→68.4%) on a synthetic constraint-dense set (S3). Layer v1.11, shot blind on the rule-only subset ofthe public ComplexBench benchmark and scored with its rule checker on whole responses, gives +3.2/+5.1points at instruction level and +5.7/+7.1 at prompt level (mappable subset: +7.4/+10.1); these subset scoresare not comparable with the benchmark’s official scores. Paired tests on per-item outcomes (McNemar;bootstrap 95% CI) make the 7B gains significant on both sets (S3 p<0.001; ComplexBench prompt-levelp=0.008) and the 3B gain significant on S3 (p=0.001) but only borderline on ComplexBench (prompt-levelp=0.08; instruction-level p=0.33). On real Chinese traffic (WildChat) the effect is near zero because explicitlyconstrained requests are rare there; the LMSYS-Chat-1M Chinese subset yielded 2 usable items and was notbuilt into a round. The first public shot (Multi-IF zh) was slightly negative (−1.8/−1.3; not distinguishable fromnoise, p=0.10/0.21) and is diagnosed as reading coverage: the extractor matched 34 of 362 mappableconstraints (type and parameter), fixed in-sample to 320/362 and out-of-sample to 145/252 on ComplexBench.Claims are bounded: feasibility is established; effectiveness is established within the shape layer; breadth isnot. Control-layer artifacts (all action implementations, rule tables and thresholds) are withheld by design; thisreport publishes effects, protocol, coverage, a failure taxonomy and boundaries.Control-layer artifacts are withheld by design (see Sections 2 and 12).

Zenodo (CERN European Organization for Nuclear Research)
Openalex Percentile: Top 13%
Natural Language Processing Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon