Toward Energy-Efficient Dynamic Resource Allocation in Uplink NOMA Systems: Deep Reinforcement Learning for Single and Multi-Cell NOMA Systems

Non-orthogonal multiple access (NOMA) is a key technology for future wireless networks, enabling improved spectral efficiency and massive device connectivity. However, optimizing power allocation, subchannel assignment, and cell association in multi-cell uplink NOMA systems is challenging due to user mobility and the NP-hard nature of the problem. This paper addresses these challenges by formulating the problem as a mixed-integer non-linear programming (MINLP) model to maximize energy efficiency (EE). We propose a deep reinforcement learning framework that employs deep Q-networks (DQN) for cell association and subchannel assignment, and twin delayed deep deterministic policy gradient (TD3) for power allocation. Simulation results reveal significant EE improvements, with multi-agent TD3 (MATD3) outperforming traditional Lagrange methods and multi-agent deep deterministic policy gradient (MADDPG). Furthermore, the proposed method exhibits robust adaptability to user mobility and superior performance in multi-cell environments, effectively mitigating inter-cell interference and enhancing resource allocation in dynamic scenarios.