Assessing the Reliability of Longitudinal Bibliometric Rankings under Incomplete Citation Histories

Longitudinal bibliometric analyses rely on annual citation histories, yet missing annual observations are frequently difficult to distinguish from true zero-citation years. We present a reproducible case-study framework for auditing citation-history gaps, tracing retrieval provenance, and quantifying the sensitivity of longitudinal indicators and rankings to explicit missing-data assumptions. The case study comprises 595 consolidated publications derived from 616 Google Scholar article records, and examines the completed calendar years 2024-2025. Three scenarios are compared: strictly observed annual counts (A), hypothetical zeros for internal gaps only (B), and hypothetical zeros for all missing years within a valid publication interval (C). The number of publications with computable growth and acceleration indicators increases from 196 (A) to 257 (B) and 490 (C); the corresponding counts for a revival indicator are 131, 152, and 177. Although the ordering of publications eligible under both compared scenarios is unchanged (Spearman and Kendall correlations equal to 1.0), complete-ranking positions and Top-K membership may change markedly. In particular, acceleration exhibits a Jaccard overlap of 0.563 at K=50 between A and C, while revival exhibits 0.667 at K=10. These findings demonstrate that unchanged indicator values and perfect correlations on a common subset do not guarantee stable rankings when eligibility changes. The zero assignments are hypothetical sensitivity assumptions, not evidence of provider semantics. The results are descriptive and based on a single researcher profile. A reusable Google Scholar Tracker software workflow supports the audit and sensitivity analysis; version 1.8.5 is publicly available on GitHub and archived on Zenodo (DOI: 10.5281/zenodo.24170285).

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-12
DOI
https://doi.org/10.48550/arxiv.2611.17233
Primary Topic
scientometrics and bibliometrics research
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Assessing the Reliability of Longitudinal Bibliometric Rankings under Incomplete Citation Histories

João Paulo Papa
arXiv (Cornell University)
scientometrics and bibliometrics research
preprint

Assessing the Reliability of Longitudinal Bibliometric Rankings under Incomplete Citation Histories

João Paulo Papa
preprint en

Abstract

Longitudinal bibliometric analyses rely on annual citation histories, yet missing annual observations are frequently difficult to distinguish from true zero-citation years. We present a reproducible case-study framework for auditing citation-history gaps, tracing retrieval provenance, and quantifying the sensitivity of longitudinal indicators and rankings to explicit missing-data assumptions. The case study comprises 595 consolidated publications derived from 616 Google Scholar article records, and examines the completed calendar years 2024-2025. Three scenarios are compared: strictly observed annual counts (A), hypothetical zeros for internal gaps only (B), and hypothetical zeros for all missing years within a valid publication interval (C). The number of publications with computable growth and acceleration indicators increases from 196 (A) to 257 (B) and 490 (C); the corresponding counts for a revival indicator are 131, 152, and 177. Although the ordering of publications eligible under both compared scenarios is unchanged (Spearman and Kendall correlations equal to 1.0), complete-ranking positions and Top-K membership may change markedly. In particular, acceleration exhibits a Jaccard overlap of 0.563 at K=50 between A and C, while revival exhibits 0.667 at K=10. These findings demonstrate that unchanged indicator values and perfect correlations on a common subset do not guarantee stable rankings when eligibility changes. The zero assignments are hypothetical sensitivity assumptions, not evidence of provider semantics. The results are descriptive and based on a single researcher profile. A reusable Google Scholar Tracker software workflow supports the audit and sensitivity analysis; version 1.8.5 is publicly available on GitHub and archived on Zenodo (DOI: 10.5281/zenodo.24170285).

arXiv (Cornell University)
scientometrics and bibliometrics research
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon