$\ell$-Step-Ahead Active Learning-Based Dual Control for Exploration and Exploitation in Auto-Optimization

In this paper, a <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell$</tex>-step-ahead active learning-based dual control for exploration and exploitation (LAL-DCEE) is proposed to address the challenges of auto-optimization amidst unknown references and environments. The algorithm of LALDCEE features a dual-loop structure, employing future gradients by looking ahead <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell$</tex>-step to guide the next control command. This approach comprises an inner loop for fast gradient updates and an outer loop for slow gradient updates. Specifically, gradients for future <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\ell$</tex>-step are iteratively refined within the inner loop using ensemble-based active learning, enabling rapid calculation of cost function gradients (i.e. estimated reference trajectory). These refined gradients then inform the outer loop, where a dual controller tailored for a system characterized by a general linear form steers the controlled system. The stability analysis of LAL-DCEE has been rigorously established. Additionally, numerical examples are employed to illustrate the effectiveness of the proposed method.