A Deep Reinforcement Learning-Based Approach for Bandwidth-Aware Service Function Chaining

Network function virtualization (NFV) is an emerging technology that is gaining popularity for network function migration. NFV converts a network function from a dedicated hardware device into a virtual network function (VNF), thereby improving the agility of network services and reducing management costs. A complex network service can be expressed as a service function chain (SFC) request, which consists of an ordered sequence of VNFs. Given the inherent heterogeneity and dynamic nature of network services, effective SFC deployment encounters significant unpredictable challenges. Machine learning-based methods offer the flexibility to predict and select the optimal next action based on existing data models. In this paper, we propose a deep reinforcement learning-based approach for bandwidth-aware service function chaining (DRL-BSFC). Aiming to simultaneously improve the acceptance ratio of SFC requests and maximize the total revenue for Internet service providers, DRL-BSFC integrates a graph convolutional network (GCN) for feature extraction of the underlying physical network, a sequence-to-sequence (Seq2Seq) model for capturing the order information of an SFC request, and a modified A3C (Asynchronous Advantage Actor–Critic) algorithm of deep reinforcement learning. To ensure efficient resource utilization and a higher acceptance ratio of SFC requests, the bandwidth cost for deploying an SFC is explicitly incorporated into the A3C’s reward function. The effectiveness and superiority of DRL-BSFC compared to the existing DRL-SFCP scheme are demonstrated via simulations. The performance measures include the acceptance ratio of SFC requests, the average bandwidth cost, the average remaining link bandwidth, and the average revenue-to-cost ratio under different SFC request arrival rates.