In-Network Q-Learning-Based Packet Forwarding for Delay Sensitive Applications

The use of Artificial Intelligence principles represents the next research challenge to support future network applications in the upcoming 6G era. In this work, we propose a novel approach: exploiting the principles of Reinforcement Learning (RL) and the availability of programmable switches to implement a new forwarding mechanism in the data plane of the 6G core network. More in detail, we define a Q-learning-based forwarding mechanism that acts at packet level and is able to select the minimum latency path at line rate. Our solution, referred to as Q-Learning-based Queue Length Routing in DAta Plane ((QL)2-RODAP), is fully decentralized and exploits in-band network telemetry to distribute network states among network nodes. We show that, either in random and real network topologies, our (QL)2-RODAP algorithm promptly reacts to sudden traffic bursts, and allows reducing the peak of queuing delays of about <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$65-85$ </tex-math></inline-formula> % with respect to other RL based approaches, thus cutting off the long tail of end-to-end latency that is critical for delay sensitive applications.