RL-Fill: Timing-Aware Fill Insertion using Reinforcement Learning

We introduce RL-Fill, a novel reinforcement learning framework for timing-aware fill insertion. RL-Fill first generates a large number of fills in the empty spaces and then removes the timing-critical fills as determined by the policy network. Towards faster convergence and stability, our framework employs a two-phase training process. In the first phase, we train the policy with offline expert data using an imitation learning scheme. In the second phase, we further optimize the policy with online data using reinforcement learning. Moreover, we propose a new data augmentation method, LayoutMix, to ensure data-efficient training despite limited number of expert data. Our results demonstrate that RL-Fill is competitive to the commercial tool and outperforms the previous machine learning-based method in timing metrics while adhering density constraints.