A Reproducible Evaluation Workflow for Quantitative Benchmarking of Medical Data Quality Tools

Propagating data defects in Electronic Health Records (EHRs) undermine the reliability of clinical AI models. However, the absence of ground-truth defect labels in real-world datasets hinders the standardized evaluation of data quality tools. This study establishes a robust evaluation workflow for a multimodal medical data quality assurance system [A], comprising controlled defect injection, real-dataset profiling, cross-tool benchmarking, and ablation studies. Experimental results demonstrate that the system achieves high detection efficacy, with a recall of 98.00% and precision of 91.91%. The system’s discriminative power is validated by a monotonic decline in quality scores as defect ratios increase. Validation on the UCI Diabetes 130 dataset showed missing-rate statistics consistent with existing literature (maximum deviation < 0.05%). Crucially, comparative analysis reveals that general-purpose tools (Great Expectations, DeepChecks, ydata-profiling) lack native medical-semantic capabilities (e.g., unit consistency) and require clean baselines, whereas our system operates effectively without such dependencies. Ablation studies further identify completeness and accuracy as the primary drivers of detection performance. This work offers a reproducible, quantitative benchmarking framework for advancing medical data quality assessment.