RAPlace: Macro Placement with Reinforcement Learning

As integrated circuits continue to scale up, placement becomes progressively more intricate. Relying on human effort for placement often takes several weeks to complete. Reinforcement learning (RL) algorithms have demonstrated potential in achieving promising placement performance in a few hours. However, existing RL-based placement methods faced limitations in their singular construction of intrinsic rewards, as they failed to simultaneously address the requirements for driving exploration and capturing the actual value of actions. In this article, we proposed a placement method based on RL, named RAPlace. Firstly, residual networks with autoencoders were introduced to encode the canvas view, coupled with a graph attention network for encoding the netlist graph to enrich state features. Secondly, RAPlace combined intuitive rewards and exploration rewards to construct a more comprehensive intrinsic reward, aiding the RL agent in better exploration and exploitation. On the ISPD2005 benchmark, RAPlace achieved a 13.59% half-perimeter wire length (HPWL) optimization compared to the RL-based method DeepTH.