Reinforcement-Learning based Resource Allocation for MEC Assisted Digital Twin Construction

This paper studies the resource allocation problem in MEC assisted digital twin (DT) construction. We consider service-based DTs and formulate the problem as a long-term optimization problem with an objective to minimize DT construction delay by optimizing spectrum and computing resource allocation on each edge node, while satisfying the loss requirements of DTs. To solve the formulated problem, we decompose the problem into multiple sub-problems, each of which can be modeled as a Markov decision process (MDP), and propose a reinforcement-learning based resource allocation (RL-DTC) algorithm to solve the MDP for DT construction. The proposed algorithm uses an Actor-Critic algorithm to perform spectrum resource and computing resource allocation for each local model training task in a DT construction process. The simulation results show that the proposed RL-DTC algorithm can effectively reduce the DT construction delay as compared with two benchmark algorithms.