When the number is right and the claim is wrong: an adversarial re-examination of a distributed coordination benchmark

A quantitative performance claim can be a valid measurement and still fail to support the conclusion drawn from it. This note documents an adversarial re-examination of the author's own strongest quantitative result, conducted under a protocol written, digested and time-stamped before any comparison was run. Four findings are reported, all adverse to the author's position and all retained: the reported advantage did not survive comparison against a grid of calibrations of a conventional alternative; a declared system property supported by a passing test proved false at protocol level; the canonical figure was not stable across execution contexts; and the benchmark design did not support the distinctions drawn from it. The note is published as a record of method rather than of result.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-09-16
DOI
https://doi.org/10.5281/zenodo.22799347
Primary Topic
Distributed systems and fault tolerance
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
article

When the number is right and the claim is wrong: an adversarial re-examination of a distributed coordination benchmark

Louis Siciliano
Zenodo (CERN European Organization for Nuclear Research)
Distributed systems and fault tolerance
article

When the number is right and the claim is wrong: an adversarial re-examination of a distributed coordination benchmark

Louis Siciliano
article en

Abstract

A quantitative performance claim can be a valid measurement and still fail to support the conclusion drawn from it. This note documents an adversarial re-examination of the author's own strongest quantitative result, conducted under a protocol written, digested and time-stamped before any comparison was run. Four findings are reported, all adverse to the author's position and all retained: the reported advantage did not survive comparison against a grid of calibrations of a conventional alternative; a declared system property supported by a passing test proved false at protocol level; the canonical figure was not stable across execution contexts; and the benchmark design did not support the distinctions drawn from it. The note is published as a record of method rather than of result.

Zenodo (CERN European Organization for Nuclear Research)
LogMeIn (United Kingdom) (GB)
Peace, Justice and strong institutions
Openalex Percentile: Top 8%
Distributed systems and fault tolerance
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

When the number is right and the claim is wrong: an adversarial re-examination of a distributed coordination benchmark — Louis Siciliano · Zenodo (CERN European Organization for Nuclear Research) (2026) | TGRS Research Map | TGRS