PPO Improvement in Different Environments

This study investigates strategies to improve the performance of the Proximal Policy Optimization (PPO) algorithm in different environments. The PPO algorithm is a widely used method in reinforcement learning, known for its robustness and efficiency in various tasks. However, its performance can vary significantly depending on the environment, and there is a continuous need for optimization to enhance its learning capabilities and overall effectiveness. The primary objective of this study is to evaluate whether specific strategies i.e. adaptive learning rate, neural network initialization, state normalization, and reward clipping can lead to better learning and eventually better overall performance of the PPO algorithm. Three distinct environments are utilized for the evaluation i.e. robotic control environments from OpenAI Gym (HalfCheetah, Walker2D) and a Futures Trading environment. Our results demonstrate that the proposed strategies lead to a PPO variant that significantly outperforms the baseline PPO in all the studied environments. These findings underscore the potential of the improved PPO performance for more effective reinforcement learning applications.