Using meta-reinforcement learning for solving the Virtual Network Embedding Problem

Introduction of Network Function Virtualization (NFV), a technique for virtualization of infrastructure on physical servers, requires careful, efficient management of substrate network resources. This has resulted in formulation of Virtual Network Embedding Problem (VNEP) for efficient deployment of virtual network functions in the substrate network. While the holder of a substrate network earns the highest revenue as the acceptance rate of customers’ embedding requests increases, they should optimize the costs for embedding the required resources. Techniques employed so far for solving VNEP often suffer from poor design, long runtime, and the need for large training sets and maintaining precision in large-scale environments. In this work, we introduce Fast Context Adaptation via Meta-Learning Virtual Network Embedding Problem (CAVIA-VNEP), a technique based on meta-reinforcement learning (MRL) as a fusion of meta-learning – capable of learning to learn, and reinforcement learning (RL). We train RL at a high computational speed with a small amount of data while using CAVIA for efficient training of neural networks . Simulation results demonstrate that CAVIA-VNEP outperforms three baseline, heuristic, and RL-based techniques in terms of long-term revenue ratio, long-term revenue-to-cost ratio, long-term acceptance ratio, and utilization of substrate nodes and links while facing challenges for reducing the average path length. • The outcomes of policy change at specific time points have not been measured in earlier studies. Therefore, we recommend using advantage functions to address this issue and reduce the effective variance. • We convert the discrete action set specified in the MDP formulation into a continuous action set and use linear regression to optimize the policy. This reformulation improves the accuracy of modeling although computation time is also increased. • At the time of composing this work, this study was the first to use a meta-learning algorithm for solving the Virtual Network Embedding Problem. • In this work, we use the Trust Region Policy Optimization (TRPO) algorithm for efficient training of our neural network value functions through monotonic improvements and controlling the policy changes.