Complexity-Aware Progressive Data Error Correction with Distilled Language Models and Conformal Reliability Control

Reliable tabular data correction is a prerequisite for trustworthy analytics in enterprise information systems. Tabular data in such environments frequently contain formatting errors, semantic conflicts, missing values, and cross-field inconsistencies that degrade downstream analytics and machine learning performance. Rule-based methods efficiently handle structural violations but miss context-dependent errors, whereas large language models (LLMs) offer strong semantic-correction capability at inference costs prohibitive for enterprise-scale deployment. This paper formulates data error correction as a progressive decision process and proposes a complexity-aware framework with three processing stages. The first stage applies deterministic rules for low-complexity structural errors. The second stage employs a task-specialized distilled language model for medium-complexity semantic correction. The third stage performs neural probabilistic–logical reasoning on a factor graph for high-complexity cross-field errors. A learnable routing mechanism assigns each record to the appropriate stage based on a lightweight complexity score. Layer-wise conformal prediction is further introduced to construct calibrated prediction sets with coverage guarantees at each stage, together with a rejection mechanism for low-confidence corrections. The framework is evaluated on one enterprise dataset and two public benchmarks (Hospital and Flights). It improves the record-level complete repair rate by 2.1 to 3.1 percentage points over the strongest baseline (GPT-4o-Direct) and by up to 16.8 points over purely rule-based repair, while reducing average inference latency by approximately 80% relative to direct GPT-4o invocation. Ablation studies confirm the critical role of complexity-aware routing and rule-trigger features, and reliability analyses show that hierarchical conformal calibration maintains tighter coverage than single-level alternatives across varying confidence requirements. These results indicate that complexity-aware progressive routing coupled with hierarchical conformal calibration provides a practical path toward high-throughput, auditable, and reliability-controlled data cleaning suitable for enterprise deployment.

Authors

Institutions

Publication Details

Journal
Mathematics
Published
2026-05-08
DOI
https://doi.org/10.3390/math14101599
Primary Topic
Data Quality and Management
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Complexity-Aware Progressive Data Error Correction with Distilled Language Models and Conformal Reliability Control

Xuejian Zhao, Enliang Wang, Jingjing Zhou, Hong Mu et al.
Mathematics
Data Quality and Management
article

Complexity-Aware Progressive Data Error Correction with Distilled Language Models and Conformal Reliability Control

Xuejian Zhao, Enliang Wang, Jingjing Zhou, Hong Mu, Chao Liu
article en

Abstract

Reliable tabular data correction is a prerequisite for trustworthy analytics in enterprise information systems. Tabular data in such environments frequently contain formatting errors, semantic conflicts, missing values, and cross-field inconsistencies that degrade downstream analytics and machine learning performance. Rule-based methods efficiently handle structural violations but miss context-dependent errors, whereas large language models (LLMs) offer strong semantic-correction capability at inference costs prohibitive for enterprise-scale deployment. This paper formulates data error correction as a progressive decision process and proposes a complexity-aware framework with three processing stages. The first stage applies deterministic rules for low-complexity structural errors. The second stage employs a task-specialized distilled language model for medium-complexity semantic correction. The third stage performs neural probabilistic–logical reasoning on a factor graph for high-complexity cross-field errors. A learnable routing mechanism assigns each record to the appropriate stage based on a lightweight complexity score. Layer-wise conformal prediction is further introduced to construct calibrated prediction sets with coverage guarantees at each stage, together with a rejection mechanism for low-confidence corrections. The framework is evaluated on one enterprise dataset and two public benchmarks (Hospital and Flights). It improves the record-level complete repair rate by 2.1 to 3.1 percentage points over the strongest baseline (GPT-4o-Direct) and by up to 16.8 points over purely rule-based repair, while reducing average inference latency by approximately 80% relative to direct GPT-4o invocation. Ablation studies confirm the critical role of complexity-aware routing and rule-trigger features, and reliability analyses show that hierarchical conformal calibration maintains tighter coverage than single-level alternatives across varying confidence requirements. These results indicate that complexity-aware progressive routing coupled with hierarchical conformal calibration provides a practical path toward high-throughput, auditable, and reliability-controlled data cleaning suitable for enterprise deployment.

MathematicsVol. 14(10)
Nanjing University of Posts and Telecommunications (CN), NARI Group (China) (CN)
Openalex Percentile: Top 3%
Data Quality and Management
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.