Evaluating Chikungunya Surveillance Reports Across Source Qualification, Model Influence, and Geographic Transfer: A Retrospective Audit Study (Preprint)

BACKGROUND Chikungunya notifications vary in case definition, geographic coverage, and reporting practice. A reported zero is not an absent report, and overall model scores may obscure influential outbreaks or differences in uncertainty across settings. OBJECTIVE To assess how original-source qualification, influential reports, geographic transfer, and prediction-interval performance affect the interpretation of models that reconstruct chikungunya surveilWe reviewed 547 annual candidate reports from 52 of 250 statistical units in a 2005–2025 search frame, distinguishing positive reports, explicit zeros, and missing records. Nine previously fitted models were compared on 85 reports from 47 units in 2022–2025 using unit-equal Poisson deviance. We examined unit-level influence, fixed-prediction sensitivity analyses, withholding schemes, strict unit-held-out transfer, and interval coverage, width, and scores. Post-inspection analyses did not refit the models.lance reports. METHODS We reviewed 547 annual candidate reports from 52 of 250 statistical units in a 2005–2025 search frame, distinguishing positive reports, explicit zeros, and missing records. Nine previously fitted models were compared on 85 reports from 47 units in 2022–2025 using unit-equal Poisson deviance. We examined unit-level influence, fixed-prediction sensitivity analyses, withholding schemes, strict unit-held-out transfer, and interval coverage, width, and scores. Post-inspection analyses did not refit the models. RESULTS Source review supported 543 of 547 reports for the source-defined annual-count target, including 226 explicit zeros. The training-unit mean had lower temporal deviance than empirical Bayes (35,800.49 vs 75,242.16), but the paired difference of −39,441.67 had a 95% unit-bootstrap interval spanning zero (−113,569.18 to 1,552.40). Brazil and Cuba accounted for 101.7% of the net difference; omitting them reversed the point ranking. Under strict unit-held-out evaluation, random forest had lower deviance (77,167) than the exploratory regional selection rules (100,884–108,611). Calibrated 95% interval coverage was 77/85 (90.6%) overall and 9/16 (56.3%) in the Americas; wider intervals increased the overall interval score from 309,178 to 378,773. CONCLUSIONS Source-defined outcomes, simple baseline models, influence diagnostics, transfer-specific validation, and geographically stratified interval evaluation are necessary for interpreting retrospective surveillance model scores. These findings support auditable comparisons of documented reports, not estimation of latent infections or prospective forecasting performance.