Self-Refinement of Reward Function by Utilizing Failure Trajectories

Designing practical reward functions in reinforcement learning is a challenging task. Traditional methods like reward shaping and Learning from Demonstration (LfD) techniques, including Imitation Learning and Inverse Reinforcement Learning, often impose high design burdens or require expert demonstrations, which may be unavailable or of poor quality. However, this study introduces an algorithm that leverages failure trajectories to self-refine the reward function, eliminating the need for external data and complex reward design. By utilizing failure trajectories, the agent learns to avoid undesirable states and actions, dynamically adjusting the reward function to promote optimal behavior. This approach not only simplifies the learning process but also enhances exploration and exploitation by transforming failures into learning opportunities, offering a promising solution to the challenges in reward function design.