A Study of Randomization-Based Improvement in CartPole

This paper studies whether reset randomization can improve the robustness of PPO in the CartPole-v1 task. CartPole is easy to solve under the default setting, but that easy setting does not always show whether a learned policy can still perform well when the starting state becomes less favorable. To address this issue, the study keeps the PPO algorithm unchanged and focuses on the environment reset process. A simple curriculum randomization strategy is used: the initial-state bound is widened gradually during training, from +/-0.05 to +/-0.10 and then to +/-0.15. The experiment compares baseline PPO and randomized PPO under a multi-seed, multi-bound evaluation design. In addition to training curves, both methods are tested under stronger reset bounds up to +/-0.40. The results show that both methods reach ceiling-level performance under mild settings, so the difference is hard to observe there. Once the evaluation becomes more difficult, the randomized method shows better robustness at part of the harder range, although the advantage is not consistent at every bound. The study suggests that randomization can be useful in CartPole, but its value depends on whether the evaluation protocol is strong enough to reveal meaningful differences.