Integrated Fault-Tolerant Attitude Control of Quadrotor Using Reinforcement Learning

This paper deals with the optimal attitude control problem for quadrotor unmanned air vehicles (UAV) with unknown nonlinear dynamics subject to component and actuator faults.The proposed integrated optimal fault tolerant control (FTC) scheme is based on reinforcement learning (RL) algorithm, without requiring prior knowledge of the system dynamics.To solve the Hamilton-Jacobi-Bellman (HJB) equation, an identifier-critic-based online RL strategy is employed with a dual neural network (NN) approximation structure.The forgetting factor in the proposed identifier update law is variable and dependent on the state estimation errors and measurement noise estimation.Choosing this variable forgetting factor increases the convergence speed and decreases the estimation error of identifier NN weights compared to the constant one while maintaining its robustness.When a fault occurs, the system continues to operate under the former control policy until the fault is detected.On the other hand, the optimal control design in the RL framework requires the initial stabilizing control condition.In order to make it possible to initiate the control learning process from the former applied FTC, this condition is relaxed by leveraging a stabilizing term in the critic update law.The Uniformly Ultimately Boundedness (UUB) of identifier and critic NN weight errors and, as a result, the convergence of the control input to the neighborhood of the optimal solution are all proved by Lyapunov theory.In the proposed method, changes in the values of faults are detected by comparing the HJB error to a predefined threshold.Finally, the simulation results are given to validate the effectiveness of the developed method.