Constrained Deep Actor-Critic Based Transmission Power Scheduling for Delay-Sensitive Applications

This paper presents an innovative downlink transmission power scheduling (TPS) scheme for the emerging delay-sensitive applications in the 6G era, focusing on enhancing the transmission efficiency while ensuring a high quality of service (QoS) for each user. Specifically, we first adopt the hard-delay constrained effective throughput and users’ packet dropout rates as performance metrics and formulate the TPS problem as a constrained Markov Decision Process (CMDP), which appropriately characterizes the requirements of delay-sensitive applications. Then, we propose a novel constrained deep Actor-Critic-based TPS (CDAC-TPS) algorithm, which can dynamically make TPS decisions by a policy network in its Actor module and evaluate the current policy by Q-networks in its Critic module without any prior information of the environment. In particular, the CDAC-TPS adopts a constrained stochastic successive convex approximation (CSSCA) method to optimize the policy network, which can better handle the stochastic non-convex objective and constraints in the TPS problem, while most of the existing policy optimization methods for CMDP are only suitable for simple convex constraints. Finally, simulation results demonstrate that the proposed TPS scheme outperforms baselines in both transmission efficiency and QoS guarantee.