Cooperative Multiagent Deep Reinforcement Learning for Computation Offloading in Digital Twin Cyber–Physical–Social System

The widespread deployment of edge computing infrastructures has greatly advanced cyber–physical–social systems (CPSSs) by enabling computation offloading for various applications. However, the heterogeneous nature of edge servers (ESs) and the single-destination offloading strategy may limit the overall network performance. In this case, tasks are exclusively handled by the administrative ES, resulting in only marginal improvements in service quality, such as energy consumption and task delay. To fully exploit the advantages of edge computing, this article investigates the cooperation among ESs within CPSS and proposes a cooperative algorithm, named MATORA, to exploit a further performance gain by calculating the task offloading and resource allocation decisions. First, we formulate the joint optimization problem as a time-varying mixed integer nonlinear programming problem; then, we utilize the multiagent deep reinforcement learning (MADRL) to learn the action policy. Second, the MATORA novelly builds reputation model for each ES to obtain insightful knowledge based on service quality, enhancing the training efficiency of MADRL. Thirdly, we simulate a digital twin-based virtual system to monitor the physical network, in which a risk-free, centralized training environment is provided for the multiagent algorithm. Finally, the simulation results and analysis directly validate the effectiveness of our algorithm.