Distributed Microservice Deployment for Satellite Edge Computing Networks: A Multi-Agent Deep Reinforcement Learning Approach

Low earth orbit (LEO) satellite networks are promising to carry out edge computing and reduce the service latency in future 6 G networks. Meanwhile, the microservice architecture provides a lightweight and flexible approach for network function deployment in satellite edge computing networks by splitting complex applications into multiple microservices. In satellite edge computing networks, a brand new problem is how to deploy various types of microservices with service function chain (SFC) constraints onto appropriate satellites to efficiently utilize limited on-board resources and reduce the service latency. In this paper, a distributed microservice deployment strategy in satellite edge computing networks is proposed by taking into account time-varying satellite network topology and diverse satellite resources. Specifically, we formulate the deployment problem as a partially observable Markov decision process (POMDP) to minimize the load imbalance as well as the service latency. A distributed multi-agent reinforcement learning (DMARL) scheme is proposed for learning a dynamic deployment strategy. Furthermore, to accommodate more microservices in SFC, a novel attention-based actor-critic strategy is designed to significantly accelerate the convergence of network training. Simulation results demonstrate that the proposed DMARL scheme can achieve a superior performance.