Exploring Variance in Reinforcement Learning Environment to Bridge the Sim-To-Real Gap: A Case Study in Color Mixing

Reinforcement Learning (RL) has proven effective in optimizing complex processes, yet its application in real-world scenarios often encounters the Sim-to-Real Gap—discrepancies in performance between simulated training and real-world deployment. This paper explores the Sim-to-Real Gap in the context of a simplified but dynamic environment: color mixing. The study focuses on training a RL agent to mix target colors from three ink colors, examining the effect of incrementally increasing simulation complexity on the agent’s ability to generalize to real-world scenarios. Motivated by the challenges in automating precision tasks like CAR-T cell therapy, this work investigates how the level of simulation detail impacts RL performance in real-world applications. The findings offer valuable insights into the role of simulation fidelity in achieving robust real-world performance, contributing to a deeper understanding of RL in high-precision, control-intensive tasks.