DCC: A Model-Free Frame to Evaluate Data Set Quality

Inspired by perturbation theory to locate thermodynamic equilibrium, we propose the data correlation convergence (DCC) framework to evaluate the quality of a data set accounting for completeness and representativeness, an alternative to conventionally computation-intensive and model-dependent approaches. The core hypothesis is that a high-quality data set should maintain stable correlation patterns under perturbations, and DCC can quantify such stability through integrating multiple correlation functions to quantify numeric correlations and distributional similarities. Based on hypothetical data sets generated by linear/determinative and random correlations, the lowest DCC was found at 10-20% linear correlations, which monotonously increased with more determinative correlations. It also revealed that a data set with around 1000 samples approaches the central limit theorem. Based on seven benchmark data sets released in material science, DCC values for the whole data set or each individual feature are informative, either to predict performance metrics including accuracies and determination coefficients for classification and regression models or to predict feature importance quantified by SHAP values. It can also efficiently compress raw data sets through recording their inherent correlation patterns and ignoring their exactly composed data. The DCC framework offers a theoretically grounded, widely applicable, and extensible approach to evaluate data set quality and annotate multiple data sets.