A dual framework for optimized data storage and retrieval using lightweight python blockchain and scalable smart contracts with IPFS

The exponential growth of IoT data demands efficient, secure, and scalable storage solutions on one hand, and efficient data migration and retrieval on the other hand are essential for the systems to be practical and acceptable for different applications. Traditional cloud-based models face latency, security, and high operational costs, while existing bi-directional data storage and retrieval-based IPFS models are not computationally efficient and incur high gas costs due to the necessary blockchain deployment. To overcome the challenges of efficient data migration, we initially developed a two-way data storage and retrieval system, as well as a scalable framework that dynamically monitors and transfers device-generated data to IPFS, records the content identifier (CID) on a blockchain, and enables secure, real-time access via smart contracts. Experimental results demonstrate that the existing work achieved an average data upload time of 117.12 seconds for a file size of 500 MB; our framework achieves a faster upload time of 7.63 seconds, marking a 93.47% improvement. We further optimize the proposed framework to reduce the file upload time incurred by the smart contracts by introducing a customizable, Python-based, blockchain-inspired, lightweight framework. It replicates the storage and retrieval functionalities of a traditional blockchain, with a file upload time of 4.2 seconds, which is 45% faster than our previous approach. This demonstrates its efficiency, security, and suitability for deployment in near-real-time and critical IoT applications, outperforming existing IPFS-smart contract-based solutions.