MONR: Multi-Objective Optimizing Network Reconfiguration Using Deep Reinforcement Learning

Modern networks require frequent configuration updates due to dynamic events, like network expansion and evolving traffic patterns. Existing network reconfiguration tools are effective in certain scenarios, but their practical deployment still has several limitations: (i) They are restricted to specific network topologies, protocols and specifications; (ii) They can cause transient violations; (iii) Their practical deployment is limited by huge computational overheads and the specialized hardware support. To address these limitations, this paper presents a Multi-Objective Optimizing Network Reconfiguration (MONR) framework, which comprises a translator and an optimizer, to automatically generate reconfiguration command sequences. The translator represents various input types into a unified graph format based on Datalog-like facts, which are regardless of the input format. Therefore, MONR supports diverse routing protocols and network specifications. The optimizer employs deep reinforcement learning techniques to simultaneously maximize the specification satisfaction of intermediate configurations and minimize both traffic shifts and the number of command updates to reduce computational overheads. It models the reconfiguration task as a multi-objective Markov Decision Process (MOMDP) and introduces a Dueling Prioritized Experience Replay Double Deep Q-Network (DPER-DDQN) algorithm to balance multiple objectives. We compare MONR with Snowcap, AED and ConfigReco. The evaluation demonstrates that MONR is 2x, 9x, and 56x faster than Snowcap, AED and ConfigReco. Furthermore, MONR maintains 100% specification consistency while reducing the traffic shifts (< 0.1) and the number of update commands (0.8 of Snowcap's).