HR²: A PVT Aware Hierarchical RL Based Sizing Framework for Robust Analog Circuit Design

As CMOS technology continues to scale, designing integrated circuits that meet performance requirements is becoming extremely challenging. In particular, designing analog circuits robust against process, voltage, and temperature (PVT) variations demands extensive manual iterations and SPICE simulations by experienced engineers. To address this challenge, we propose HR<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup>, a hierarchical reinforcement learning (HRL) based circuit design framework that targets robust analog design across all PVT corners. HR<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> divides circuit sizing problems into several sub-problems and utilizes a multi-agent HRL structure, allowing agents to specialize in their respective sub-problems and to become aware of variations. We also introduce a lightweight variation modeling technique within the RL environment to embed PVT considerations efficiently. The reward functions that consider worst-case variations and design margins are also developed to enhance the robustness of the resulting designs. HR<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> achieves up to 76% reduction in sample count, 74% decrease in runtime, while showing 55× lower Monte Carlo simulation fail ratio compared to previous RL based approaches.