Validation of the Kythera Labs all-payer real-world data asset: a provenance-anchored nine-block framework for tokenized evidence generation

Kythera Labs is one of the largest all-payer, tokenized administrative claims real-world data (RWD) assets in the United States, comprising 12.5 billion medical claims and 12.9 billion pharmacy claims contributed by more than 500 payer organisations and refreshed weekly. In this paper all-payer denotes adjudicated claims from commercial, Medicare and Medicaid payers rather than a single plan or benefit type; tokenized denotes that direct identifiers are irreversibly hashed into encrypted Datavant tokens at the data originator, behind the HIPAA boundary; and distributed through denotes that Columbia Data Analytics licenses and delivers the asset under a Data Use Agreement without re-adjudicating source claims. The source universe contains 315,817,351 unique patient tokens accrued cumulatively across 2015–2025; this is not a point-in-time census, and we do not claim contemporaneous coverage of the insured U.S. population. Despite growing use of such assets for regulatory, health-economics and payer decision-making, no pre-registered, open-source validation of the Kythera asset has been published to date. We pre-specified a nine-block validation framework addressing internal consistency, concurrent validity (framed here as external consistency), construct validity, representativeness, missingness, record linkage, inter-coder reliability, temporal stability and predictive validity, and applied it to an analytic cohort of 38,947,221 patients with ≥ 2 longitudinal observations and ≥ 1 tokenized linkage element between 1 January 2015 and 31 July 2025. Seventeen pre-registered equations operationalise the framework. The nine analytic blocks are anchored to an explicit appraisal of data provenance and intake quality assurance so that the data-generation process, and not only derived statistics, is documented. Record-linkage performance was estimated in a blinded, clinician-adjudicated probability sample of 11,882 candidate pairs with pre-specified subgroup stratification, plus a purposive sample of 2,400 pairs representing four difficult matching scenarios. Predictive models were evaluated under random, temporal and geographic validation schemes. All analysis code is released open-source under MIT licence at https://github.com/obaser3791/kythera-validation . Internal consistency was strong for multi-item claims-derived constructs (Charlson comorbidity composite α = 0.87; Elixhauser comorbidity composite α = 0.89); because internal consistency is not a sufficient reliability criterion for count-type comorbidity indices, we added 12-month test–retest reliability (intraclass correlation 0.72–0.85), known-group validity and discrimination for 12-month mortality (c = 0.617–0.793). External consistency against NHANES and MEPS yielded a mean absolute percentage error (MAPE) of 2.8% for diabetes prevalence and ≤ 4.7% across six chronic conditions. A two-factor social-determinants confirmatory factor model reached CFI = 0.971, TLI = 0.958, RMSEA = 0.038 and SRMR = 0.034 and fitted better than a one-factor alternative (CFI = 0.902; RMSEA = 0.071). Standardized mean differences against the 2023 American Community Survey were ≤ 0.061 across all 14 demographic and geographic strata evaluated. Fellegi–Sunter tokenized linkage achieved a 96.2% match rate with estimated false-match rate 0.003; sensitivity was 96.7% (95% CI 96.3–97.1) and specificity 99.5% (99.3–99.7) against blinded adjudication, with sensitivity ranging from 94.8% (Medicaid) to 97.6% (Medicare) across subgroups and falling to 91.9–94.0% in difficult matching scenarios. Inter-coder reliability on ICD-10-CM diagnoses yielded Cohen’s κ = 0.84 (condition-specific κ 0.71–0.92; positive predictive value 0.82–0.96). Predictive validity reached AUROC = 0.834 for 90-day mortality and 0.791 for 30-day readmission in a held-out random split, 0.821 and 0.779 in a temporal holdout, and 0.812–0.845 and 0.768–0.803 across leave-one-region-out folds. The Kythera Labs asset met every pre-registered threshold across nine data-quality dimensions, supporting its use as a starting point for question-specific fitness-for-purpose assessment in regulatory, HEOR and payer research. Because fitness-for-purpose is defined relative to a specific research question, exposure, outcome and decision context, these results should not be read as certifying the database as universally fit for regulatory use, and several blocks evaluate derived statistics rather than the validity of individual clinical events. The nine-block framework presented here is reusable by sponsors, regulators and academic investigators for fit-for-purpose qualification of similar all-payer RWD assets.