Enhancing the Reliability of Control Systems Using an Improved Deep Reinforcement Learning Framework

This paper presents an improved framework for deep reinforcement learning algorithms integrating online system identification, based on the Dyna-Q architecture. The proposed framework is designed to tackle the challenges of both Multi Input Multi Output (MIMO) and Multi Input Single Output (MISO) systems in complex, industry relevant environments, thereby significantly enhancing adaptability and reliability in industrial control systems. It should be noted that in the suggested novel framework, the system identification and model control processes run in parallel with the control process, ensuring a reliable backup in case of faults or disruptions. To verify the efficiency of the aforementioned approach, comparative evaluations in the presence of three of the most common deep reinforcement learning algorithms, i.e. Deep Q Network (DQN), Deep Deterministic Policy Gradient (DDPG), and Twin Delayed Deep Deterministic Policy Gradient (TD3), are conducted on industry-relevant environments simulations available in OpenAI Gym, including the Cart Pole, Pendulum, and Bipedal Walker, each chosen to reflect specific aspects of the novel framework. Results demonstrate that the proposed method for leveraging both real and simulated experiences in this framework improves sample efficiency, stability, and robustness.