Hierarchical Integration of Reinforcement Learning and Optimization Algorithms for Time-Efficient Design Automation of Complex Analog Circuit

Design automation of complex analog circuits (CAC) with multiple sub-blocks is challenging mainly due to large design search space, uncertain intermediate subgoal creation, and lengthy CAC simulation runtime. In this work, we propose a hierarchical and heterogeneous integration framework as a fully automated and time-efficient CAC design optimization solution. In Particularly, we (i) decompose CAC into two levels hierarchically and for the first time introduce hierarchical RL agents with hindsight and subgoal testing to automate the subgoal creation between these two levels. The subgoal converges to the optimal value through algorithm interactions. (ii) We enable high-level design space dimensionality reduction, minimize CAC simulation runs through a buffer hold, and employ low-level sub-block execution parallelization to reduce overall runtime. (iii) We construct a heterogeneous integration of different RL algorithms and black-box optimization algorithms in hierarchy to further boost the speed by benefiting both from the hierarchical structure and the advantages of each different algorithm. Experiments on four CAC topologies demonstrate that this framework achieves a maximum of 11.4× speed up compared to existing methods at the desired figure-of-merit. This work opens up a time efficient design automation route for complex analog circuits and systems.