GARF: A Self-supervised Data Cleaning System with SeqGAN

High-quality data is essential for data science and machine learning applications, but unfortunately, real-world data often contains significant amounts of errors, such as typos, missing values, and data inconsistencies. Despite all the efforts in cleaning data using either logical or learning-based methods, in practice, data cleaning still requires high human cost, for either manually providing data repairing rules or preparing labeled datasets for training machine learning models. In this paper, we introduce GARF, a novel data cleaning system based on sequence generative adversarial networks (SeqGAN). One key information GARF tries to learn is data repair rules. To automatically extracts data repair rules from dirty data, GARF employs a SeqGAN to capture the dependency relationships, and converts the information learned by machine to interpretable data repair rules for humans. Additionally, considering that both generated rules and data may not be fully trusted, GARF provides a co-cleaning process to iteratively update inaccurate rules and repair dirty data until there is no tuple violating rules. We have implemented and deployed GARF as an open-sourced system, and demonstrated its usability on data cleaning in real-world scenarios.