Discrete Soft Actor-Critic Algorithm With Heuristic-Based Action Mapping for RMSCA in MCF-EONs

This paper proposes a novel deep reinforcement learning (DRL) architecture that enables effective decoupling between the agent and the environment for routing, modulation, spectrum, and core allocation (RMSCA) in multi-core fiber elastic optical networks (MCF-EONs). In the architecture, a heuristic-based action mapping layer (HAM) is designed between the agent and the environment. This layer maps the diverse action spaces of MCF-EONs into a unified and efficient space, providing the agent a stable and consistent interface. The HAM employs heuristic rules to filter and rank all possible decision options, ultimately selecting the top <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H$</tex-math></inline-formula> high-quality candidate solutions for the agent to make decisions. Meanwhile, a general linear regression (LR) method is introduced to dynamically compute an optimal action space size <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H$</tex-math></inline-formula> tailored to the specific scenario, improving the system's flexibility and robustness across varying conditions. Finally, a reward function combining spectrum fragmentation and link load is designed to guide the agent in efficiently considering the state of spatial resource utilization. The proposed algorithm is evaluated under two different network topologies, various multi-core fibers, and traffic load conditions. The results show that, compared with advanced heuristic algorithms and DRL approaches, the proposed method reduces blocking probabilities by up to 89% and 83%, respectively, and demonstrates excellent generalization performance.