Hyperfuzzing: black-box security hypertesting with a grey-box fuzzer

Abstract Despite being a severe error where programs inadvertently reveal confidential information, insecure flows rarely receive explicit attention during software testing. LeakFuzzer uses an input-output non-interference property, specialised via a security flow policy for the program under test, to advance the state of the art. It detects insecure flows by using hypertesting for violations of the program’s non-interference property. LeakFuzzer extends the capabilities of the state of the art fuzzer, AFL++, and thus inherits its advantages such as scalability, automated input generation, high coverage and low developer intervention. It can thus detect the same set of errors as AFL++, as well as being able to detect violations of secure information flow policies at small additional performance costs. This offers a significant advance in scalability and automation for the state of the art. We evaluated LeakFuzzer on a diverse set of 12 C and C++ benchmarks containing known bugs that cause confidential information to be disclosed, ranging in size from just 80 to over 900k lines of code. Nine of these are taken from real-world CVEs including Heartbleed and a recent error in PostgreSQL. Given 20 24-hour runs, LeakFuzzer can find 100% of the insecure flows in the SUTs whereas existing techniques using the CBMC model checker and AFL++ augmented with different sanitizers can only find 40% at best.