Stratégie de contrôle pour la gestion de pannes d'actionneurs par apprentissage par renforcement profond : Application aux véhicules sous-marins autonomes

Autonomous dynamic systems, such as exploration drones, are at risk from actuator faults, which can compromise the operation or lead to a system loss. This is why it is necessary to establish a Fault-Tolerant Control (FTC) strategy to maintain system performance and stability under actuator faults. Traditional FTC methods commonly use control reallocation, redistributing forces among remaining actuators. Control reallocation requires explicit fault diagnosis, i.e. identifying the fault’s origin and intensity in real time. Explicit fault diagnosis becomes impractical with multiple faults or limited computational capacity. These limitations have led to the exploration of Deep Reinforcement Learning (DRL) for system control. The idea of existing DRL-based FTC strategies in the literature is to leverage DRL to directly generate control inputs for the actuators, allowing the system to autonomously adapt to faults. DRL offers high adaptability for actuator fault recovery, enabling the control strategy to operate without explicit fault diagnosis. However, DRL based control lacks stability guarantees due to the unpredictability of DRL model outputs, which poses a challenge in guaranteeing stability of the system. This thesis addresses this challenge by aiming to demonstrate that the combination of DRL and model-based control improves the performance and stability of an autonomous dynamic system subject to an actuator fault compared to existing methods. To support this statement, the chosen methodology involves performing control real location using DRL within a model-based control framework, integrating DRL’s adaptability with the stability benefits of model-based methods. DRL redistributes forces by adjusting the Control Allocation Matrix, with local stability ensured through recalculated control gains in a model-based framework. Trained in simulation to address varied fault scenarios, the DRL model provides real-time control reallocation in the case of faults. This method is applied to a BlueROV2 Heavy, which is an autonomous underwater vehicle equipped with multiple thrusters, in both simulation and real-world experiments. Results demonstrate the effectiveness of DRL-based control reallocation for thruster faults in the BlueROV2 Heavy, outperforming non-model-based and non-DRL adaptive methods. Additionally, the results highlight the DRL model’s generalisation capacity, as it successfully manages fault scenarios not encountered during training. These findings support a promising FTC approach to actuator fault tolerance that combines DRL’s adaptability with the stability of model-based control, without any need for explicit fault diagnosis.