Homogeneous Multi-Chip PDN Optimization for DDR Memory Architecture Using RL

In this paper, we propose a reinforcement learning (RL)-based framework to address hierarchical feature variations, multi-chip interactions, and partial decoupling capacitor (decap) allocation across hierarchical power distribution network (PDN) levels, including the PCB, PKG, and chip domains. We formulate the problem as a multi-decap placement problem (MultiDPP), aimed at optimizing the location and allocation of decaps to achieve three critical objectives: minimizing PDN impedance, reducing cost, and optimizing chip area utilization. The RL framework employs a dual-reward mechanism combining a stepwise reward (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{R}_{\text{step }}$</tex>) for immediate action evaluation and an episode-wise reward (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{R}_{\boldsymbol{e p}}$</tex>) to achieve overall decap minimization. To validate the proposed approach, we apply it within a DDR5 environment, demonstrating significant reductions in design time, overall decap usage, and SSN compared with traditional methods. The results confirm that accounting for hierarchical differences and multi-chip interactions yields a robust, scalable solution for on-chip PDN design, paving the way for more efficient and reliable DIMM architectures in current and future memory technologies.