Model-based reinforcement learning for hydraulic systems: a PILCO-based control strategy for controlling an inverted hydraulic pendulum

Abstract In fluid power systems, hydraulic drives are widely used but often rely on linear controllers that require precise tuning to specific operational contexts. Traditional control methods, while effective, can be inflexible and struggle with the nonlinear aspects of hydraulic systems, such as fluid compressibility and friction. Experimental investigations to understand these systems are often impractical due to cost and safety concerns, making simulations a valuable alternative. However, accurate simulations are computationally intensive and not always feasible for real-time control. This work introduces a Probabilistic Inference for Learning and Control (PILCO) algorithm to address these challenges. PILCO is a model-based reinforcement learning (RL) algorithm that constructs a dynamics model from observed data, incorporating uncertainties to predict future system behavior with minimal environmental interactions. The algorithm was applied to control a physical inverted hydraulic pendulum, a complex task due to the system’s nonlinearities. PILCO successfully learned precise control policies with only 64 seconds of interaction time, demonstrating the effectiveness of reward shaping and offline data initialization in accelerating learning. The results highlight PILCO’s potential to advance optimal control in real-world hydraulic systems through efficient and adaptable RL strategies.