Analyzing the impact of elusive faults on blockchain reliability

Blockchain has recently become very popular due to its use in cryptocurrencies and potential application in various domains (e.g., retail, healthcare, and insurance). The smart contract is a key part of blockchain systems and specifies an agreement between transaction participants. Nowadays, smart contracts are being deployed to carry residual faults, including severe vulnerabilities that lead to different types of failures at runtime. Fault detection tools can be used to detect faults that may then be removed from the code before deployment. However, in the case of smart contracts, the common opinion is that tools are immature and ineffective. In this work, we carry out a fault injection campaign to empirically analyze the runtime impact that realistic faults present in smart contracts may have on the reliability of blockchain systems. We pay particular attention to the faults that elude popular smart contract verification tools and show if and in which ways the faults lead the blockchain system to fail at runtime. We map the observations to the fault detection capabilities of three state-of-the-art fault detection tools, namely Mythril, Slither, and Securify. The results show that the tools individually have poor detection capabilities (e.g., Securify with 6.4% accuracy and Mythril with 60% accuracy) or tend to generate false alerts (i.e., only 1.74% of Slither's alerts are correct). The results also show several elusive faults responsible for severe blockchain failures, such as A_MCV, which impacts the integrity of the ledger, and I_MVMSV, which causes gas depletion, just to name a few.