Scalable and efficient on-chain data management in blockchain for large biomedical data

Blockchain technology is gaining traction in the biomedical sector due to its ability to improve trust and reduce the risk of fraud and errors in health data management. However, the large volume of biomedical datasets has slowed its adoption due to poor scalability. This challenge is especially relevant for applications that rely on blockchain's strong immutability by storing data directly on-chain. In this work, we demonstrate the potential of blockchain to create a secure and trustless environment for managing large on-chain records. Specifically, we detail an efficient, index-based approach for storing data on the Ethereum blockchain. We show that insertion and retrieval speeds remain nearly constant relative to database size, scaling linearly with the amount of data processed. Additionally, we achieve substantial efficiency gains through low-level assembly optimizations on the Ethereum Virtual Machine, highlighting the limitations of the Solidity compiler. Finally, we illustrate this approach through a practical case study, by designing and implementing a smart contract for storing and querying training certificates on the Ethereum blockchain. Our solution achieves 2x faster data insertion, 500x faster retrieval, 60% lower gas costs, and 50% lower storage usage compared to baseline methods. It won first place for track 1 of the 2022 iDASH secure genome analysis competition. We also demonstrate that this solution readily adapts to other data types, enabling efficient on-chain storage and retrieval of text, RNA-seq, or biomedical image data.