Attention-Enhanced Rainbow DQN for Cell Reselection in Satellite Communication Networks

In satellite communication networks, user devices can be in either active or idle states. In active states, devices actively communicate with specific cells, maintaining a continuous connection for data transmission. In idle states, however, user devices do not actively communicate with specific cells but need to monitor potential cells and perform cell reselection (CR) under appropriate conditions for future connections. Inefficient CR decision-making is a significant issue in these states, particularly in low Earth orbit (LEO) satellite networks characterized by large-scale and highly dynamic environments. To address the inefficiency in CR decision-making under these conditions, especially when dealing with a varying number of candidate satellites, we first model the reselection problem as a Markov decision process. By incorporating a self-attention mechanism for candidate satellite correlation modeling and adjusting the related network structure, we propose an Attention-Enhanced Rainbow DQN algorithm. This algorithm improves the model's learning ability and decision stability in tasks involving a varying number of candidate satellites. Simulations demonstrate that the proposed algorithm outperforms weighted decision algorithms and conventional Rainbow DQN algorithms in terms of load balancing, signal-to-noise ratio (SNR), and the number of reselections.