Adaptive Control Strategies for Networked Systems: A Reinforcement Learning-Based Approach

Advances in industrial 5G communication technologies and robotics create new possibilities while also increasing the complexity and variability of networked control systems. The additional throughput and lower latency provided by 5G networks enable applications such as teleoperation of machinery, flexible reconfigurable robotic manufacturing cells, or automated guided vehicles. These use cases are set up in dynamic network environments where communication latency and jitter become critical factors that must be managed. Despite the advancements in 5G technologies, such as ultra-reliable low-latency communication (URLLC), adaptive control strategies such as reinforcement learning (RL) remain critical to handle unpredictable network conditions and ensure optimal system performance in real-world industrial applications. In this paper, we investigate the potential of RL in scenarios with communication latency similar to a public 5G deployment. Our study includes an incremental improvement by utilizing long short-term memory-based neural networks in combination with proximal policy optimization in this scenario. Our findings indicate that incorporating latency into the training environment enhances the robustness and efficiency of RL controllers, especially in scenarios characterized by variable network delays. This exploration provides insights into the feasibility of using RL for networked control systems and underscores the importance of incorporating realistic network conditions into the training phase.