Joint Two Stage Beamforming and User Scheduling for LEO Satellite Communications via Hierarchical Multi-Armed Bandit

This paper proposes a joint two-stage beamforming and user scheduling scheme based on the multi-armed bandit (MAB) for massive multiple-input multiple-output low earth orbit (LEO) satellite systems. Given the impracticality of availing instantaneous channel state information (CSI) due to significant latency issues, we propose to obtain statistical CSI (S-CSI) using MAB historical beam training data, and use the obtained S-CSI to achieve beamforming and user scheduling simultaneously, without the need for dedicated pilot resources for S-CSI estimation. Specifically, in the first stage, the analog beamforming problem incorporated with user scheduling for maximizing the received energy is formulated as a combinatorial MAB problem, with the action space encompassing both the beam space and the user space. Then, an action selection strategy called dual upper confidence bound is proposed to reduce problem complexity and achieve faster convergence. This strategy consists of two layers: the upper layer focuses on beam selection, while the lower layer manages user scheduling within the selected beams. Furthermore, we prove that the proposed strategy achieves logarithmic regret, which means the action converges to beams and users with the largest average received energy. In the second stage, based on historical user received signals, a multi-user digital precoder is developed to mitigate interference among users sharing the same time-frequency resources. Simulations demonstrate its superior performance.