A Distributional Reinforcement Learning-Based Strategy for Pod Scheduling in Satellite Clusters

An effective Pod scheduling strategy is crucial for maintaining optimal performance and resource utilization in distributed satellite clusters. The inherent complexity of the satellite cluster environment, combined with the sequential decision-making nature of Pod scheduling, makes it challenging to develop a strategy that meets multiple optimization objectives. Previous studies addressing multi-objective Pod scheduling often rely on DQN-based algorithms that model cumulative returns as expectation values <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(V(s)\ \mathbf{or}\ Q(s,\ a.))$</tex> leading to significant loss of detailed distributional information. Furthermore, traditional DQN methods struggle with the dynamic variability of multi-objective reward functions, making weighted summation an unstable approach for achieving load balance in satellite clusters. To overcome these limitations, we propose a novel Pod scheduling strategy based on distributional reinforcement learning (DRL), which models the distribution <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$Z(s,\ a)$</tex> of cumulative returns, thereby preserving detailed distributional information. To enhance learning in satellite cluster environments, we employ multi-dimensional reward functions capturing the joint distributions of source-specific rewards. Our strategy effectively integrates optimization objectives including CPU, memory, inter-satellite links, and load balancing, ensuring robust and efficient service quality. Experimental results demonstrate that our proposed strategy significantly improves scheduling success rates, resource utilization efficiency, and load balancing in complex satellite cluster environments, outperforming traditional scheduling algorithms and DQN-based methods.