Dynamic Overlay Selection in SD-WAN Using Reinforcement Learning: A Collaborative Multi-Agent Approach

This thesis presents a novel framework for adaptive SD‑WAN overlay selection using Multi‑Agent Reinforcement Learning (MARL) under the Centralized Training with Decentralized Execution (CTDE) paradigm. We model each branch as an agent that competes for shared transport overlays, and we introduce a tunable λ parameter to encode heterogeneous Service‑Level Agreement (SLA) priorities directly into the reward function. Our custom Gymnasium environment incorporates stochastic request arrivals, per‑request queue dynamics and bandwidth depletion. We compare value‑based (DQN) and policy‑gradient (PPO) agents trained on a joint action space of four discrete overlay decisions, using detailed logging callbacks to track per‑episode returns, overlay congestion rates, and joint‑action distributions. Experimental results across diverse traffic and SLA scenarios illustrate convergence trends, demonstrate SLA compliance, while λ effectively trades off branch priorities. This work advances intelligent overlay selection in SD-WAN by combining MARL coordination, SLA-aware reward structuring, and a traffic-aware simulation environment based on queueing dynamics.