Improving Timing & Power Trade-off in Post-place Optimization Using Multi-agent Reinforcement Learning

In recent years, post-place optimization has emerged as a critical stage in physical design, aiming to improve power, performance, and area (PPA). Among various optimization techniques, buffer insertion, gate sizing, and Vth assignment have become leading optimization techniques for decades. However, these techniques are traditionally applied sequentially, leading to a critical suboptimality problem. Each technique obtains its own iterations performed step by step, preventing the best-optimal optimization for a target instance. To address this limitation, we propose a novel reinforcement learning (RL) based post-place optimization framework that performs various optimization techniques simultaneously. Moreover, to overcome the persistent headache of chip design, timing and power trade-off, we employ multiple agents that target each specific objective. By leveraging deep RL and graph neural network (GNN), our framework models optimal policies, dynamically selecting the most effective action given a target instance. Consequently, our framework obtained an improved Pareto-frontier set compared to comparison baselines while exhibiting 80% and 43% improvements in total negative slack and leakage power, respectively. The results demonstrate that our method outperforms weighted-sum-based co-optimization methods in optimizing timing and power.