A DDPG Hybrid With Graph Scheme for Resource Management in Digital Twin-Assisted Biomedical Cyber-Physical Systems

In this paper, we focus on the resource optimization issue in Wireless Body Area Networks (WBANs) section of a novel Biomedical Cyber Physical System (BM-CPS) in which the Digital Twin (DT) technique is adopted concurrently. We propose a scenario where multiple physiological sensor nodes continuously monitor the electrophysiology signals from patients and connect to a Cyber Managing Center (Cyb-MC) in a wireless way to transmit the physiological indices to the paramedic reliably. Specifically, to optimize the energy efficiency of physiological sensors while ensuring the reliability of emergency-critical electrophysiology signals transmissions and modeling the uncertain stochastic environments, the optimization issue is transformed into a Markov Decision Process (MDP) in terms of joint transmission mode, relay assignment, transmission power and time slot scheduling consideration. Subsequently, we propose a Random Graph-enabled Deep Deterministic Policy Gradient (RG-DDPG) scheme to tackle the challenge while releasing computing complexity. In addition, as an innovative paradigm for reshaping cyberspace applications, the DT of WBAN is created to capture the time-varying resource status, where virtualized and intelligent resource management can be performed uniformly. Finally, extensive simulation studies verified the advancement of the proposed scheme and demonstrated that the DT can mirror the topology, predict the behavior, and administrate the resources of the WBANs.