Adaptive Control for A Full-Order Unmanned Undersea System Based on Reinforcement Learning

Unlike general reduced-order control methods, the control method introduced in this article is grounded directly on the full-order controlled model of an unmanned undersea system. Through stability analysis, the range of values for the control parameters is determined. Recognizing the inherent adaptability of the traditional policy-based reinforcement learning, which targets the action conditional probability distribution function (CPDF) as its learning objective, this article extends this notion and introduces a parameter adaptive adjustment method based on reinforcement learning. The method is based on the classic A2C framework and achieves adaptive adjustment of control parameters by learning the conditional probability distribution function of the parameter. While the method boasts the advantages of higher intelligence and wider applicability compared to classical adaptive methods, it also exhibits superior learning efficiency compared to classical reinforcement learning due to the predefined range of control parameter values based on stability analysis. Through the numerical simulation, the article validates the effectiveness of the introduced method and demonstrates its uniqueness by comparing it with an adaptive control method based on fuzzy logic.