2D to 3D Placement for Monolithic Systems using Reinforcement Learning with Dynamic Hierarchical Cluster Assignment

Monolithic 3D (M3D) ICs represent a significant advancement in VLSI technology, and unfortunately introduce a new dimension to the traditional, already NP-complete, 2D placement problem. A promising approach is to convert 2D placement into 3D placement using tier partitioning algorithms. Existing tier partitioning algorithms do not leverage historical data, such as prior placement configurations and performance metrics, that could improve efficiency and quality of placement. Utilizing historical data can help refine decision-making processes, reduce computational overhead, and enhance the overall effectiveness of tier partitioning strategies. To allow reuse of historical data, we propose a novel approach for M3D placement based on models and Reinforcement Learning (RL) that we name Dynamic Hierarchical Cluster Assignment with RL (DHCARL), which simplifies the search process by forming dynamic hierarchical clusters to preserve 2D placement when assigning clusters to 3D tiers. This approach uses a small set of parameters that enable effective reuse of historical data. With a Machine Learning (ML) model, low-code AutoML libraries help evaluate and select an RL policy. We propose novel Placement-aware graph networks (PAGN) to capture design connectivity and placement patterns in placement representation. We tested DHCARL on a diverse set of benchmarks, ranging from multiplexers (8- to 128-bit MuxShifter) to large-scale designs, including PicoRV32A, TATE-Pairing, and JPEG Encoder, implemented in 2-tiered and 4-tiered Monolithic 3D technology. Experimental results demonstrate that the proposed Reinforcement Learning-based Dynamic Hierarchical Assignment (DHCARL) approach outperforms state-of-the-art methods like TP-GNN and ART-3D, achieving a 6.3% lower placement cost on TATE-Pairing and 5.9% on PicoRV32A with similar runtime as TP-GNN.