Enhanced NSA-DRX Mechanism for Cognitive 5G Networks Utilizing Q-Learning and Long Short Term Rewards

Managing the growing energy demands of user equipment (UE) in 5G networks is critical for sustaining prolonged device standby times. The Discontinuous Reception (DRX) mechanism plays a pivotal role in reducing power consumption, but its implementation often results in a trade-off with increased packet delays. Addressing these challenges, we propose an enhanced NSA-DRX mechanism tailored for the NSA deployment model. This mechanism integrates both <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RRC_CONNECTED</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">RRC_IDLE</i> states, modeled using an eight-state semi-Markov process to capture the stochastic and dynamic state transitions inherent in DRX operations. The proposed mechanism employs cognitive principles by integrating the Q-learning algorithm to optimize short sleep cycles dynamically. Additionally, a Long Short Term Reward (LSTR) framework is introduced, leveraging multi-scale rewards to balance power efficiency and delay. Simulations demonstrate that the Q-learning and LSTR-enhanced NSA-DRX mechanism achieves a 10% improvement in power-saving performance compared to traditional models while maintaining a balanced trade-off with delay. These results highlight the efficacy of reinforcement learning and reward-based optimization in advancing cognitive communication and networking systems for next-generation networks.