Reinforcement Learning with World Models for Autonomous Excavation Optimization in Wheel Loaders

Automating the bucket-filling task in wheel loaders is challenging due to the complex, nonlinear interaction between the bucket and granular material. This work presents a model-based reinforcement learning approach to optimize the bucket-filling strategy for Zeux, Volvo’s autonomous electric wheel loader concept. A Long Short-Term Memory (LSTM) surrogate model is trained on data from Volvo’s high-fidelity simulator to emulate realistic dynamics, enabling efficient policy training using Proximal Policy Optimization (PPO) with imagined rollouts. This reduces computational cost and eliminates the need for direct interaction with the high-fidelity simulator. Compared to Volvo’s current rule-based driver model, the learned policy achieves 89% improvement in productivity and 56% increase in energy efficiency. Our results show that world models can accelerate reinforcement learning for heavy machinery control, enabling the discovery of strategies that outperform controllers based on human expert behavior.