PLIMSOLL Pre-Registration and Attestation Specification, Version 1

A reported machine learning evaluation result carries no evidence about theprocess that produced it. A reader cannot determine whether the acceptancethreshold was fixed before the result was observed, whether the evaluationdataset is unchanged, or how many attempts preceded the reported one. Thelast question is the most consequential: a party may evaluate repeatedlyunder varying conditions and report only the attempt that satisfied theirclaim, without falsifying any individual number. This deposit specifies a mechanism that makes such process facts checkable bya third party who trusts neither the claimant nor the operator of theverification service. Before execution, a seal containing the decision rule,threshold, and digests identifying the dataset and system under test iscanonicalized, signed, and appended to a public Merkle tree. After execution,an attestation binds observed aggregate values to that seal. The log, not theclient, assigns each attestation a monotonically increasing attempt numberscoped to its seal, so the number of attempts made against a pre-registrationis a published fact rather than an optional disclosure. The specification defines: canonical serialization under RFC 8785 JCS withUnicode NFC and line-ending normalization; a dataset digest computed over anunordered multiset of canonicalized rows; a decision-rule grammar with noarithmetic, function calls, or variables; nearest-rank percentile semantics;fixed-precision decimal comparison from original lexical form, never throughbinary64; RFC 6962 Merkle construction with domain-separated hashing andEd25519 signed tree heads; log-assigned attempt numbering; and asupersession mechanism that permits disclosure but not amendment. A conforming implementation transmits only digests and metadata. Datasets,models, prompts, and outputs are never received, stored, or transmitted. The accompanying technical note documents the construction and states itslimitations explicitly, including that digest-to-artifact correspondencecannot be verified by a system that declines to receive the artifact, that asingle-operator log admits equivocation absent a gossip protocol, and that noempirical study of the underlying phenomenon has yet been conducted. Released under CC0 1.0 to permit unrestricted implementation. A conformancechecklist for independent implementations is included.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-08-26
DOI
https://doi.org/10.5281/zenodo.22107451
Primary Topic
Adversarial Robustness in Machine Learning
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

PLIMSOLL Pre-Registration and Attestation Specification, Version 1

Gautam Khosla
Zenodo (CERN European Organization for Nuclear Research)
Adversarial Robustness in Machine Learning
article

PLIMSOLL Pre-Registration and Attestation Specification, Version 1

Gautam Khosla
article en

Abstract

A reported machine learning evaluation result carries no evidence about theprocess that produced it. A reader cannot determine whether the acceptancethreshold was fixed before the result was observed, whether the evaluationdataset is unchanged, or how many attempts preceded the reported one. Thelast question is the most consequential: a party may evaluate repeatedlyunder varying conditions and report only the attempt that satisfied theirclaim, without falsifying any individual number. This deposit specifies a mechanism that makes such process facts checkable bya third party who trusts neither the claimant nor the operator of theverification service. Before execution, a seal containing the decision rule,threshold, and digests identifying the dataset and system under test iscanonicalized, signed, and appended to a public Merkle tree. After execution,an attestation binds observed aggregate values to that seal. The log, not theclient, assigns each attestation a monotonically increasing attempt numberscoped to its seal, so the number of attempts made against a pre-registrationis a published fact rather than an optional disclosure. The specification defines: canonical serialization under RFC 8785 JCS withUnicode NFC and line-ending normalization; a dataset digest computed over anunordered multiset of canonicalized rows; a decision-rule grammar with noarithmetic, function calls, or variables; nearest-rank percentile semantics;fixed-precision decimal comparison from original lexical form, never throughbinary64; RFC 6962 Merkle construction with domain-separated hashing andEd25519 signed tree heads; log-assigned attempt numbering; and asupersession mechanism that permits disclosure but not amendment. A conforming implementation transmits only digests and metadata. Datasets,models, prompts, and outputs are never received, stored, or transmitted. The accompanying technical note documents the construction and states itslimitations explicitly, including that digest-to-artifact correspondencecannot be verified by a system that declines to receive the artifact, that asingle-operator log admits equivocation absent a gossip protocol, and that noempirical study of the underlying phenomenon has yet been conducted. Released under CC0 1.0 to permit unrestricted implementation. A conformancechecklist for independent implementations is included.

Zenodo (CERN European Organization for Nuclear Research)
Peace, Justice and strong institutions
Openalex Percentile: Top 7%
Adversarial Robustness in Machine Learning
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.