Enhancing Proximal Policy Optimization in the Pendulum-v1 Environment Through Clustering-Based State Space Simplification

This study aims to enhance the performance of the Proximal Policy Optimization (PPO) algorithm in continuous control tasks, particularly in the Pendulum-v1 environment. We introduce a clustering-based approach to simplify the state-action space, improving learning efficiency and maximizing cumulative rewards. The state-action pairs are clustered using k-means, emphasizing regions with higher rewards to guide the agent's policy optimization. Experimental results demonstrate that clustering significantly improves PPO's performance, with the 10 -cluster configuration yielding the best outcomes in terms of reward consistency and efficiency. After tuning, the clustering-enhanced PPO achieves near-optimal performance, outperforming the baseline PPO by a wide margin.