Auditable verification of bibliographic reference integrity under incomplete provider metadata: evidence traces, selective abstention, and structured author diagnostic boundaries

Abstract Bibliographic-integrity audits must distinguish incorrect references from incomplete provider evidence. A frozen selective verifier was evaluated on 3,712 reference strings from 80 source articles and an independently labelled, balanced natural-validation sample of 600 references (575 valid, 15 inconsistent, 10 unresolved; inter-annotator κ = 0.6594). Under provider-degraded execution , 542/542 API-queried sample rows recorded at least one provider failure. Endpoint A covered 398/590 reference-standard-decisive cases (67.46%), with TP/TN/FP/FN = 9/134/255/0 and 3.41% precision; Endpoint B routed all 25 review-positive cases but required human review of 466/600 references. A post-hoc selective-routing probe reduced false positives from 255 to 40, at the cost of reducing decisive coverage from 398/590 to 179/590; human review workload did not decrease. In a separate 320-reference error-enriched set, surname-level author-score decomposition produced no incremental decision changes on four natural author-identity errors. In a DOI-omitted 90-reference cohort (30 conference papers, 30 chapters, 30 theses), the frozen method covered 58/88 decisive cases with five true positives, one false positive, and two false negatives; a within-cohort parser patch increased coverage to 63/88 without reducing false negatives. A bounded 14-pair live-provider comparison recorded six different selected DOIs, seven unchanged DOIs, and one case without a comparable DOI; direct DOI insertion into corrected strings prevents causal attribution of the selected-DOI changes. The evidence supports evidence-traced review routing as an auditable 1 workflow, while demonstrating the limits of definitive automatic inconsistency claims, provider availability, and within-cohort parser improvements. MSC Classification: 68P20 JEL Classification: C80

Authors

Institutions

Publication Details

Journal
Research Square
Published
2026-11-13
DOI
https://doi.org/10.21203/rs.3.rs-11423167/v1
Primary Topic
Data Quality and Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Auditable verification of bibliographic reference integrity under incomplete provider metadata: evidence traces, selective abstention, and structured author diagnostic boundaries

Ercan Erkalkan
2 links
Research Square
Data Quality and Management
preprint

Auditable verification of bibliographic reference integrity under incomplete provider metadata: evidence traces, selective abstention, and structured author diagnostic boundaries

Ercan Erkalkan
preprint en

Abstract

Abstract Bibliographic-integrity audits must distinguish incorrect references from incomplete provider evidence. A frozen selective verifier was evaluated on 3,712 reference strings from 80 source articles and an independently labelled, balanced natural-validation sample of 600 references (575 valid, 15 inconsistent, 10 unresolved; inter-annotator κ = 0.6594). Under provider-degraded execution , 542/542 API-queried sample rows recorded at least one provider failure. Endpoint A covered 398/590 reference-standard-decisive cases (67.46%), with TP/TN/FP/FN = 9/134/255/0 and 3.41% precision; Endpoint B routed all 25 review-positive cases but required human review of 466/600 references. A post-hoc selective-routing probe reduced false positives from 255 to 40, at the cost of reducing decisive coverage from 398/590 to 179/590; human review workload did not decrease. In a separate 320-reference error-enriched set, surname-level author-score decomposition produced no incremental decision changes on four natural author-identity errors. In a DOI-omitted 90-reference cohort (30 conference papers, 30 chapters, 30 theses), the frozen method covered 58/88 decisive cases with five true positives, one false positive, and two false negatives; a within-cohort parser patch increased coverage to 63/88 without reducing false negatives. A bounded 14-pair live-provider comparison recorded six different selected DOIs, seven unchanged DOIs, and one case without a comparable DOI; direct DOI insertion into corrected strings prevents causal attribution of the selected-DOI changes. The evidence supports evidence-traced review routing as an auditable 1 workflow, while demonstrating the limits of definitive automatic inconsistency claims, provider availability, and within-cohort parser improvements. MSC Classification: 68P20 JEL Classification: C80

Research Square
Marmara University (TR)
Data Quality and Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon