Deep Reinforcement Learning for QoT-Aware Routing, Modulation, and Spectrum Assignment in Elastic Optical Networks

Deep reinforcement learning (DRL) has been applied to the routing, modulation, and spectrum assignment (RMSA) in elastic optical networks (EON), enabling the learning of RMSA policies through interaction between a DRL agent and the EON environment. Existing approaches aim to make decisions that fulfill spectrum utilization requirements. For the quality of transmission (QoT) assurance, they rely on distance-dependent modulation selection. However, other factors such as physical impairments, spectrum fragmentation, and traffic dynamics can impact QoT. In this work, we introduce DeepRMSA-QoT (deep reinforcement learning for QoT-focused routing, modulation, and spectrum assignment), a technique using the DRL framework to explore efficient QoT-aware RMSA policies for EONs. We propose a state representation that includes QoT-level information. Additionally, two reward functions are developed alongside the basic one, incorporating the QoT information. Considering QoT during the development of state representation and reward functions can guide the agent to actions that meet the QoT requirements and minimize exploration blindness when performing RMSA. This ultimately leads to more efficient learning of improved policies. Furthermore, a QoT unit is designed to guarantee the QoT and to facilitate the implementation of proposed reward functions. Extensive simulation results demonstrate that our proposed approach outperforms the existing approaches. Our approach can identify lightpath feasibility based on QoT requirements, select feasible lightpaths to meet QoT of demands, and avoid infeasible ones. In addition, our approach achieves a lower blocking rate while provisioning high bit-rates with dynamic traffic demands in different topologies.