Reinforcement Learning-Based Optimal Control for Continuous-Time Quantized Systems via Adaptive Dynamic Programming

Ahstract- This paper presents a comprehensive investigation into the stability analysis and design conditions for quantizers to ensure the closed-loop stability of continuous-time linear quantized systems, under the conditions derived by the small-gain theorem. The study begins by deriving explicit bounds on the quantizer parameters required for maintaining system stability. Building on this foundation, an optimal controller is designed using the linear quadratic regulator (LQR) framework, providing an efficient data-driven control strategy. To further enhance the system's performance, an adaptive dynamic programming (ADP) algorithm, referred to as the hybrid iteration (HI) method, is developed. This algorithm effectively learns the optimal control policy by leveraging the trajectories of the quantized states and inputs, thereby addressing the challenges posed by quantization constraints. The proposed HI approach combines the advantages of adaptive learning and optimization, making it well-suited for continuous-time systems with limited information. The simulation results confirm that the ADP approach with the provided conditions not only stabilizes the quantized system but also achieves optimal control performance under the specified quantization conditions. This study offers valuable insights and a robust methodological framework for addressing stability and control challenges, with insights to be expanded to continuous-time nonlinear quantized systems, with potential applications in various engineering domains, such as networked systems, robotics and autonomous systems.