Prediction is not selection: a pre-registered study of design-time mechanism choice in author name disambiguation

Can a designer measure a property of their own data before building anything and learn which classification mechanism to build? On seven public author name disambiguation datasets the answer is mostly no, for two separate reasons. The first is structural. A rule assigning on an exact surface match is a strict sub-case of a retriever assigning from the single nearest surface form whenever the similarity function gives its maximum only to identical strings, because a test string present in the history is then its own nearest neighbour. In the comparison most practitioners would run, the rule therefore cannot win at any cutoff, and we measured zero rule wins on 199,713 held-out author mentions in two metric families before we understood why. A wider form of the argument says where a precision-first rule cannot win. The second reason is empirical. On a redesigned pair in which neither mechanism contains the other, an exact-evidence rule against a k-nearest vote, neither pre-registered signal is actionable at the margin we fixed in advance once the seven datasets are pooled: the mechanisms sit half an accuracy point apart, 0.8244 against 0.8294, no band on either axis clears the margin, and both selection hypotheses return Inconclusive. The pooled result is mostly inspire's, which holds four fifths of the blocks, and it is not uniform. Two of the five datasets large enough to judge are decided at that margin in opposite directions: kisti for the rule by 0.1131, through better matching, and zbmath for the vote by 0.0631, through the rule's abstentions. One unregistered signal, block size, does carry a decided pooled verdict, and it is composition and abstention rather than matching quality. The consistency statistic, by contrast, predicts each mechanism's own accuracy at Spearman 0.51 and 0.83. Prediction is not selection.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-13
DOI
https://doi.org/10.5281/zenodo.24184393
Primary Topic
scientometrics and bibliometrics research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Prediction is not selection: a pre-registered study of design-time mechanism choice in author name disambiguation

Maitreya Sapariya
Zenodo (CERN European Organization for Nuclear Research)
scientometrics and bibliometrics research
preprint

Prediction is not selection: a pre-registered study of design-time mechanism choice in author name disambiguation

Maitreya Sapariya
preprint en

Abstract

Can a designer measure a property of their own data before building anything and learn which classification mechanism to build? On seven public author name disambiguation datasets the answer is mostly no, for two separate reasons. The first is structural. A rule assigning on an exact surface match is a strict sub-case of a retriever assigning from the single nearest surface form whenever the similarity function gives its maximum only to identical strings, because a test string present in the history is then its own nearest neighbour. In the comparison most practitioners would run, the rule therefore cannot win at any cutoff, and we measured zero rule wins on 199,713 held-out author mentions in two metric families before we understood why. A wider form of the argument says where a precision-first rule cannot win. The second reason is empirical. On a redesigned pair in which neither mechanism contains the other, an exact-evidence rule against a k-nearest vote, neither pre-registered signal is actionable at the margin we fixed in advance once the seven datasets are pooled: the mechanisms sit half an accuracy point apart, 0.8244 against 0.8294, no band on either axis clears the margin, and both selection hypotheses return Inconclusive. The pooled result is mostly inspire's, which holds four fifths of the blocks, and it is not uniform. Two of the five datasets large enough to judge are decided at that margin in opposite directions: kisti for the rule by 0.1131, through better matching, and zbmath for the vote by 0.0631, through the rule's abstentions. One unregistered signal, block size, does carry a decided pooled verdict, and it is composition and abstention rather than matching quality. The consistency statistic, by contrast, predicts each mechanism's own accuracy at Spearman 0.51 and 0.83. Prediction is not selection.

Zenodo (CERN European Organization for Nuclear Research)
The Charutar Vidya Mandal (CVM) University (IN)
scientometrics and bibliometrics research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon