Gated Reward Weighting for Single-Policy Reinforcement Learning Control of a Valve-Controlled Hydraulic Inverted Pendulum
Proportional valve-controlled cylinders are the most widely used and most representative electro-hydraulic drives. This paper considers such a drive moving an inverted pendulum on a cart: one controller must swing the pendulum up, stabilize it, and track a cart reference, while the valve flow is nonlinear and the three tasks compete for the same valve. A single reinforcement learning (RL) policy based on deep deterministic policy gradient (DDPG) performs all three tasks. During training, a gated, state-dependent reward weighting shifts the emphasis from pendulum stabilization to cart tracking once the pendulum has been captured. In simulation, the controller is compared with a staged energy-based swing-up and linear quadratic regulator (LQR) baseline and with an RL controller trained with fixed reward weights. The proposed controller swings the pendulum up in 2.0 s instead of 4.7 s and completes a 0.5 m/s reference at which both comparison controllers reach the end stop. It also tracks more accurately than the fixed-weight controller, and without the gate the training does not balance the pendulum at all. The costs are a constant tracking offset of 10 mm, more valve motion, and a larger pendulum excursion than the baseline under an external force.
Authors
- Bobo Helian (ORCID: https://orcid.org/0000-0003-2484-5676)
- Litong Lyu (ORCID: https://orcid.org/0000-0002-1658-4448)
- Penghui Li
Institutions
- Karlsruhe Institute of Technology (DE)
- Shijiazhuang Tiedao University (CN)
Publication Details
- Journal
- Actuators
- Published
- 2026-11-18
- DOI
- https://doi.org/10.3390/act15110612
- Primary Topic
- Reinforcement Learning in Robotics
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China
- Department of Education of Hebei Province