CCS-MASAC Resource Allocation Method for Collaborative Cluster Satellite Systems in 6G

The collaborative cluster satellite system (CCS) within the 6G network establishes the foundation for robust services in the future Star-Earth integrated network by coordinating multiple low-Earth orbit (LEO) satellites for collaborative observation missions and efficient space mission processing. This paper proposes a model-based soft actor-critic (SAC) algorithm, CCS-MASAC, for optimizing throughput in clustered satellite systems within 6G networks. The algorithm integrates the clustering degree of CCS with the entropy regularization term in SAC, proposing an adaptive adjustment method. Unlike existing studies, in this work, we adopt an environment model-based policy optimization approach for the first time. Model-based policy optimization focuses on improving the sample efficiency of reinforcement learning algorithms. It allows agents to learn iteratively in both real and simulated environments, which improves sample efficiency, convergence, and algorithm robustness. To address the dimensionality explosion in single-agent reinforcement learning (RL) algorithms, we extend this approach to a multi-agent RL algorithm by defining observable neighborhoods for each agent, further enhancing performance. Simulation results indicate that the CCS-MASAC algorithm proposed in this paper enhances throughput by 15–20% and accelerates convergence by 30% compared to existing algorithms, including the multi-agent deep Q-network (MADQN), multi-agent proximal policy optimization (MAPPO), multi-agent deep deterministic policy gradient (MADDPG) and multi-agent double and dueling deep Q-learning (MAD3QL). The scalability and robustness of the algorithms are verified by scalability experiments and experiments under dynamic channel conditions. This research provides new solutions for throughput optimization and resource management in CCS systems.