TIDE: A Configuration-Driven Framework for Editing and Imputation of Hierarchical Administrative and Survey Data

Hierarchical administrative and survey datasets—tax filings linked to schedules, business surveys linked to establishments, household surveys linked to persons—must be validated and imputed before they can support official statistics. This paper presents TIDE (Tax Imputation & Data Engine), a configuration-driven framework that applies the Fellegi–Holt minimum-change paradigm to forms linked one-to-one and one-to-many without flattening them. Domain knowledge lives in YAML configurations, not in the engine. For each record, TIDE localizes the minimum-weight set of fields whose values can be changed to satisfy every edit rule at once, checking this imputability condition directly rather than through implied edits; it imputes those fields with methods that a tournament on held-out data selects per field; and it restores accounting identities by moving only the localized fields, jointly, to the nearest consistent values. Records that could be resolved only by overwriting reported values that no evidence identifies as wrong are routed to an editor-review queue rather than forced. On a 100,000-return synthetic tax dataset, TIDE changes 95 percent of the corrupted values, routes 115 returns (0.12 percent) to review, rarely changes correctly reported values, and keeps a nearly constant processing time per return: 5.0 milliseconds at 100,000 returns against 6.1 at 5,000.

Authors

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-11-21
DOI
https://doi.org/10.5281/zenodo.24408983
Primary Topic
Data Quality and Management
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

TIDE: A Configuration-Driven Framework for Editing and Imputation of Hierarchical Administrative and Survey Data

Mark Xu
Zenodo (CERN European Organization for Nuclear Research)
Data Quality and Management
preprint

TIDE: A Configuration-Driven Framework for Editing and Imputation of Hierarchical Administrative and Survey Data

Mark Xu
preprint en

Abstract

Hierarchical administrative and survey datasets—tax filings linked to schedules, business surveys linked to establishments, household surveys linked to persons—must be validated and imputed before they can support official statistics. This paper presents TIDE (Tax Imputation & Data Engine), a configuration-driven framework that applies the Fellegi–Holt minimum-change paradigm to forms linked one-to-one and one-to-many without flattening them. Domain knowledge lives in YAML configurations, not in the engine. For each record, TIDE localizes the minimum-weight set of fields whose values can be changed to satisfy every edit rule at once, checking this imputability condition directly rather than through implied edits; it imputes those fields with methods that a tournament on held-out data selects per field; and it restores accounting identities by moving only the localized fields, jointly, to the nearest consistent values. Records that could be resolved only by overwriting reported values that no evidence identifies as wrong are routed to an editor-review queue rather than forced. On a 100,000-return synthetic tax dataset, TIDE changes 95 percent of the corrupted values, routes 115 returns (0.12 percent) to review, rarely changes correctly reported values, and keeps a nearly constant processing time per return: 5.0 milliseconds at 100,000 returns against 6.1 at 5,000.

Zenodo (CERN European Organization for Nuclear Research)
Data Quality and Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon