A lightweight, easy-integration reward shaping study for progress maximization in Reinforcement Learning for autonomous driving

This paper addresses the challenge of sample efficiency in reinforcement learning (RL) for autonomous driving, a domain characterized by long-term dependencies and complex environments. While RL has shown success in various fields, its application to autonomous driving is hindered by the need for numerous samples to learn effective policies. We propose a novel, lightweight reward-shaping method called room-of-adjust to maximize learning progress. This approach separates rewards into continuous tendency rewards for long-term guidance and discrete milestone rewards for short-term exploration. Our method is designed to be easily integrated with other approaches, such as efficient representation, imitation learning, and transfer learning. We evaluate our approach on a hill-climbing task with uneven surfaces, which simulates the spatial-temporal reasoning required in autonomous driving. Results show that our room-of-adjust reward shaping achieves near-human performance (81.93%), whereas other reward shaping and progress maximization methods struggle. When combined with imitation learning, the performance matches human levels (97.00%). The Study also explores the method's effectiveness in formulating control theory, such as 4-wheel independent drive (4WID) systems. With reduced spatial-temporal reasoning, reward shaping can match human performance (89.7%). However, control theory cannot be trained together with complicated spatial-temporal progress maximization.