PALO BlueprintBench: An Evidence-Grounded Methodology for Evaluating Specification Fidelity in AI-Generated Business Blueprints
AI-generated business specifications can preserve the vocab-ulary of a source document while changing an obligation, permission, exception, or decision authority. Evaluating this transformation requires separating syntactic conformity, se-mantic fidelity, execution reliability, and the human work needed to repair an output. We present PALO Blueprint-Bench, an open benchmark module and evaluation method-ology for this problem. The method binds source scope, raw outputs, proposition annotations, configurations, and accep-tance policies to inspectable evidence. It distinguishes crit-ical source-rule inversions, unsupported critical additions, required-proposition coverage, and correction effort, without collapsing them into an unvalidated composite score. We re-port four diagnostic evidence cohorts: historical adapter tests on three passages containing 4,994 UTF-8 bytes; an inter-nal procurement pilot with 18 model configurations, 324 as-signed calls, and three synthetic cases from one authored fam-ily; a six-scenario feasibility cycle with 48 provisional propo-sitions and 60 requests to three configurations; and a real-document generation study on three public procurement doc-uments totalling 200 pages, in which 14 of 27 assigned calls returned a contract-valid specification.
Authors
- Fabrizio Degni (ORCID: https://orcid.org/0009-0006-7900-9359)
Institutions
- Politecnico di Milano (IT)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-12
- DOI
- https://doi.org/10.5281/zenodo.24151879
- Primary Topic
- Artificial Intelligence in Law
- Type
- article
- Field-Weighted Citation Impact
- 0.00