PLIMSOLL Pre-Registration and Attestation Specification, Version 1
A reported machine learning evaluation result carries no evidence about theprocess that produced it. A reader cannot determine whether the acceptancethreshold was fixed before the result was observed, whether the evaluationdataset is unchanged, or how many attempts preceded the reported one. Thelast question is the most consequential: a party may evaluate repeatedlyunder varying conditions and report only the attempt that satisfied theirclaim, without falsifying any individual number. This deposit specifies a mechanism that makes such process facts checkable bya third party who trusts neither the claimant nor the operator of theverification service. Before execution, a seal containing the decision rule,threshold, and digests identifying the dataset and system under test iscanonicalized, signed, and appended to a public Merkle tree. After execution,an attestation binds observed aggregate values to that seal. The log, not theclient, assigns each attestation a monotonically increasing attempt numberscoped to its seal, so the number of attempts made against a pre-registrationis a published fact rather than an optional disclosure. The specification defines: canonical serialization under RFC 8785 JCS withUnicode NFC and line-ending normalization; a dataset digest computed over anunordered multiset of canonicalized rows; a decision-rule grammar with noarithmetic, function calls, or variables; nearest-rank percentile semantics;fixed-precision decimal comparison from original lexical form, never throughbinary64; RFC 6962 Merkle construction with domain-separated hashing andEd25519 signed tree heads; log-assigned attempt numbering; and asupersession mechanism that permits disclosure but not amendment. A conforming implementation transmits only digests and metadata. Datasets,models, prompts, and outputs are never received, stored, or transmitted. The accompanying technical note documents the construction and states itslimitations explicitly, including that digest-to-artifact correspondencecannot be verified by a system that declines to receive the artifact, that asingle-operator log admits equivocation absent a gossip protocol, and that noempirical study of the underlying phenomenon has yet been conducted. Released under CC0 1.0 to permit unrestricted implementation. A conformancechecklist for independent implementations is included.
Authors
- Gautam Khosla
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-08-26
- DOI
- https://doi.org/10.5281/zenodo.22107450
- Primary Topic
- Adversarial Robustness in Machine Learning
- Type
- article
- Field-Weighted Citation Impact
- 0.00