On the Applicability of Benford’s Law to Detect Saturation in Fuzzing (Registered Report)

Knowing when a fuzzing campaign has reached saturation is crucial for practitioners to avoid unnecessarily lengthy campaigns without missing bugs within given resources. Unfortunately, existing solutions for determining the saturation point rely on coverage measurements, which are often error-prone and unreliable. In this paper, we present a novel way to detect saturation in fuzzing based on Benford's law, which describes the characteristics of naturally occurring numbers. Specifically, we hypothesize that repeatedly occurring numbers in the fuzzing process, such as the number of mutated bytes, should show specific numerical patterns dictated by Benford's law when the fuzzer reaches saturation, thereby the fuzzing process becomes less biased (hence, more natural). The key observation is that grey-box fuzzers become less biased as in random black-box testing when they reach saturation because there will be no seed to prioritize. We aim to test our hypothesis on 29 real-world programs using the state-of-the-art fuzzer, AFL++, and empirically show that one can use Benford's law to detect fuzzing saturation.