Robot Learning Optimal Control via an Adaptive Critic Reservoir

In this paper we develop a reservoir-based adaptive critic framework for optimal real-time control of nonlinear robotic systems. Unlike traditional neural network (NN) based critics, deterministic reservoir computing significantly reduces computational complexity by requiring only the training of output weights. Using deterministic reservoir computing, the method efficiently approximates the value function and computes optimal control laws, achieving rapid error convergence and computational efficiency. In addition, a momentum-enhanced adaptation law is proposed to accelerate convergence rates. The framework also contributes to robot learning by enabling adaptive behavior through iterative policy improvement, allowing robots to autonomously refine their control strategies by self-learning. It is theoretically validated under the persistency of excitation condition and demonstrated through simulations of trajectory tracking compared against traditional NN parameterization.