Olympia: Fuzzer Benchmarking for Solidity

Over the last few years, smart-contract hacks have resulted in the loss of billions of assets. To efficiently identify such vulnerabilities, academic and industrial researchers have developed several popular smart-contract fuzzers. However, it has been challenging to objectively compare their bug-finding effectiveness. In this paper, we present Olympia, the first benchmark-generation tool that is designed for smart-contract, rather than general-purpose, fuzzers. We have used Olympia to evaluate the effectiveness of four well known, open-source fuzzers for Solidity smart contracts.