MADRL Based Energy-Efficient Handover for Direct-to-Cell Ultra-Dense LEO Satellite Network

The ultra-dense low earth orbit satellite network (UD-LSN) is composed of numerous satellites with different orbital parameters, providing dense overlapping coverage to smartphone-type user equipment (UE) via direct-to-cell technology. The selection of the handover (HO) target and UL transmission power control after HO can significantly impact the energy consumption and quality of service (QoS) of UE. To this end, this paper proposes a user-centric energy-efficient HO (UCEEH) scheme based on multi-agent dueling double deep recurrent Q-network (MAD3RQN), which jointly optimizes the HO target selection and the transmission power of UE, while considering both uplink and downlink objectives and constraints to minimize UE energy consumption and ensure the QoS. A user-centric HO control framework based on UE's independent observation is designed without increasing additional energy consumption for training the UCEEH. To better balance the exploration and exploitation of historical decision experiences, a utility-referenced heuristic experience accumulation (UHEA) algorithm is proposed. The simulation results indicate that the UCEEH algorithm with UHEA can significantly reduce the energy consumption of UE while ensuring the highest QoS.