TIDE: A Configuration-Driven Framework for Editing and Imputation of Hierarchical Administrative and Survey Data
Hierarchical administrative and survey datasets—tax filings linked to schedules, business surveys linked to establishments, household surveys linked to persons—must be validated and imputed before they can support official statistics. This paper presents TIDE (Tax Imputation & Data Engine), a configuration-driven framework that applies the Fellegi–Holt minimum-change paradigm to forms linked one-to-one and one-to-many without flattening them. Domain knowledge lives in YAML configurations, not in the engine. For each record, TIDE localizes the minimum-weight set of fields whose values can be changed to satisfy every edit rule at once, checking this imputability condition directly rather than through implied edits; it imputes those fields with methods that a tournament on held-out data selects per field; and it restores accounting identities by moving only the localized fields, jointly, to the nearest consistent values. Records that could be resolved only by overwriting reported values that no evidence identifies as wrong are routed to an editor-review queue rather than forced. On a 100,000-return synthetic tax dataset, TIDE changes 95 percent of the corrupted values, routes 115 returns (0.12 percent) to review, rarely changes correctly reported values, and keeps a nearly constant processing time per return: 5.0 milliseconds at 100,000 returns against 6.1 at 5,000.
Authors
- Mark Xu (ORCID: https://orcid.org/0009-0009-9285-9421)
Publication Details
- Journal
- Zenodo (CERN European Organization for Nuclear Research)
- Published
- 2026-11-21
- DOI
- https://doi.org/10.5281/zenodo.24408983
- Primary Topic
- Data Quality and Management
- Type
- preprint