Blockchain-Aided Mobile Edge Intelligence With Sparse Rewarded Computing Resource Allocation

In the blockchain-aided mobile edge intelligence (B-MEI) network, base stations (BSs) each with an edge server not only execute tasks offloaded from users but also serve as blockchain nodes to ensure trustworthy data management of task execution in a decentralized manner. A challenging issue of B-MEI is that, both task execution and blockchain consensus demand significant computing resources. Insufficient resource allocation inevitably leads to resource wastage and even degrades system performance, especially in scenarios where task arrivals and wireless channels are dynamically changing. Driven by this issue, we first formulate the dynamic allocation of computing resources as a Markov Decision Process (MDP) to maximize the average transaction throughput of the B-MEI system. In particular, we find that, due to the time-consuming nature of blockchain consensus, the MDP rewards stemmed from the block generation is typically delayed and sparse. To address this issue, we propose a novel reinforcement learning algorithm, named Stable Hindsight Credit Assignment (SHCA), to effectively learn the stable credit assignment for sparse rewards across intermediate actions to optimize policy decisions. Simulation results demonstrate that the proposed algorithm improves the average transaction throughput by 78% compared with the baseline algorithm.