Improving DDPG-Based Reinforcement Learning for Efficient Microservice Deployment in Edge Environments

Microservice-based edge computing is a promising computing paradigm. It decomposes an application as a collection of small independent microservices and deploys microservices on the edge of network. This computing paradigm offers better service deployment and execution in comparison with the monolithic-based cloud computing paradigm. This paper addresses the multi-objective microservice deployment problem (MOMDP) in edge computing. MOMDP aims to minimize the total microservice interaction traffic cost in edge computing while maximizing the edge resource utilization. We first formulize MOMDP as a nonlinear integer programming (NLIP) problem. For efficiently tackling this problem, we apply for the deep deterministic policy gradient (DDPG) algorithm which is a widely used algorithm in the field of reinforcement learning. To maintain the above two objectives while not violating the edge resource constraint, we propose two resource allocation algorithms to incorporate expert-based knowledge into DDPG. Finally, we perform extensive simulation experiments to demonstrate the effectiveness of the proposed approach.