<scp>SnapCC</scp> : Effective File System Consistency Testing Using Systematic State Exploration

Modern file systems have become increasingly feature-rich and highly complex, making crash consistency increasingly difficult to perform correctly. Thoroughly testing file systems for crash consistency bugs, however, is difficult to achieve good results due to insufficient state exploration, a lack of guidance for test case generation, and missing support for modern file system features. In this article, we present a new approach towards testing file system consistency: systematic file system persistent state exploration. In contrast to previous efforts, our design addresses these shortcomings through testing the crash consistency property of file systems systematically using the following procedures. Initially, we use system call generation and execution feedback from fuzzers to generate workloads that stress the file system code. During this process, we systematically explore all possible persistent states of the underlying file system for the given workload and subsequently use them as file system image inputs for the crash recovery routines to produce a corresponding file system state. After the file system finishes processing an image input, we deploy an efficient file system checker to compare the contents of the image to that of a correct image and determine whether the image is inconsistent, consequently determining whether we have triggered a crash consistency bug in the underlying file system. We implemented a prototype tool SnapCC and deployed it for testing multiple mainstream file systems on Linux. We compared its effectiveness along with other relevant tools Hydra and B3, where our results show that SnapCC achieves 16% to 44% better coverage over Hydra, and finds 15 new consistency bugs, whereas B3 and Hydra find 2 and 6, over a period of 2 weeks, further demonstrating SnapCC’s effectiveness in discovering file system consistency bugs. To demonstrate our approach’s adaptability, we also tested SnapCC on 5 other file systems, upon which 7 additional bugs were found.