Step-wise Execution of Data-Centric Systems

Data-centric systems play a crucial role in computer systems, and their correctness is of paramount importance. Logic bugs are one of the main types of bugs in data-centric systems, leading to incorrect results. Existing methods for testing data-centric systems face challenges such as ineffective test oracles, insufficient coverage of target functionalities, and low efficiency in test case generation. To address these issues, we propose a general, novel black-box methodology for testing various kinds of important data-centric systems. Our key insight is that we can incrementally generate the test case and use the intermediate results to construct the reference test case. The result of the original test case should be identical to that of the reference test case. We have applied this methodology to test both Datalog engines and DBMSs, successfully uncovering 75 unique bugs. We believe that this idea might be applicable to many other systems as well, and will next explore this idea to Deep Learning libraries.