Topology and Parameter Optimization of High-Order Δ-Σ Modulators Towards Superior Efficiency and Stability with Multi-Agent Reinforcement Learning

The Δ-Σ analog-to-digital converter (ADC), with the modulator as its core component, has posed considerable challenges to the designers, due to the complex topologies and instability issues. Thanks to reinforcement learning (RL), appropriate models can be trained to automatically generate efficient modulator structures without the need for prior datasets. Proximal policy optimization (PPO), one of the latest and most promising branches of RL, can be optimally used by virtue of its simplicity and less hyperparameter tuning. This study focuses on multi-agent PPO (MAPPO) for the design of high-order Δ-Σ modulators, with two agents handling topology and parameter optimization respectively in a cooperative way. We address the challenge of both efficiency and stability through proper mathematical formulation and effective integration of weighted objectives. Through extensive simulations and iterative processes of MAPPO, our proposed methodology demonstrates effectiveness in maximizing the efficiency and stability objectives of the desired Δ-Σ modulator via a reward mechanism.