Data Sinks Deployment for Backbone-Assisted Real-Time PD-NOMA Networks based on Reinforcement Learning

Real-time performance is one of the most vital metrics in Backbone-assisted Power-Domain Non-Orthogonal Multiple Access Wireless Networks(BA-PDNOMAWNs) for Industrial Internet of Things applications. Since the relative geographic relationship between data sinks and wireless sensors affects real-time performance greatly, setting reasonable locations for data sinks is thus a feasible way to achieve high real-time performance for the scenarios where wireless sensors are not mobile. In this paper, we model the problem of finding the optimal locations for given sinks in BA-PDNOMAWNs so as to minimize uplink transmission latency. To find a low-complexity algorithm, we first formalize the problem as a Markov Decision Process, and then propose a reinforcement learning algorithm based on the classic Multi-Agent Deep Deterministic Policy Gradient, where its reward function is designed elaborately to accelerate convergences. Experimental results show that multiple data sinks in the BAPDNOMAWNs can reduce uplink transmission latency by $2.2 \times$ compared to that with a single data sink. Additionally, compared to the classic Simulated Annealing optimization algorithm and the other siting algorithms, such as Clustering Center and Grid Partitioning in BA-PDNOMAWNs, our proposed algorithm can even reduce the uplink transmission delay by $2 \times$, thus achieving better real-time performance.