Multi-Agent Deep Reinforcement Learning for Task Offloading in LEO Satellite Networks

In recent years, the rapid development of Low Earth Orbit (LEO) satellite networks has highlighted the need for enhanced data processing capabilities. However, the limited computational capacity and inherent heterogeneity of LEO satellites pose significant challenges for efficient task computation, such as minimizing latency and energy consumption. To address these challenges, we propose QMIX-sat, a novel multi-agent reinforcement learning algorithm. We transform the optimization problem into a decentralized partially observable Markov decision process (Dec-POMDP) and solve it using an enhanced version of the QMIX algorithm. In the satellite scenario under study, we introduce attention modules and residual connections to generate accurate task segmentation and allocation decisions, effectively mitigating the issues caused by high-dimensional observation spaces, such as inter-satellite and satellite-ground link states. Additionally, given that QMIX is commonly used in discrete action spaces, this work extends QMIX to continuous action spaces, ensuring precise task segmentation and allocation. We construct a LEO satellite network model for experimental verification, and the results demonstrate that QMIX-sat significantly improves task allocation efficiency and overall performance within LEO satellite networks compared to existing algorithms.