Performance and scalability testing for blockchain consensus protocols: an empirical framework

Blockchain unlocks several use cases leveraging unique characteristics of trustless peer-to-peer transactions and decentralised computing. Reaching consensus in blockchains is crucial to ensure a consistent state of the ledger across the network participants. However, consensus also introduces performance and scalability gaps concerning traditional centralised systems. Understanding those gaps is paramount, especially if blockchains are established as the main computing infrastructure for future digital services. In this paper, we propose an empirical approach to performance and scalability testing in blockchain systems. We provide a framework that unlocks measurements of well-established metrics under different configuration scenarios. This framework establishes a systematic methodology based on simulated private blockchain networks. To this extent, we first provide a standardisation for performance and scalability metrics. Then, we describe the testing methodology that establishes a reproducible environment for running quantitative tests with efficient, asynchronous, results computation. We implement the framework with two blockchain platforms using different consensus protocols, respectively Proof of Work and Proof of Stake. We demonstrate that the performance of both protocols is impacted by different configuration settings like the difficulty parameter in Proof of Work and the gossip protocol in Proof of Stake.