A novel deep reinforcement learning framework fusing knowledge and probabilistic modeling for prognostic and maintenance of aircraft engines

Abstract The accurate prediction of remaining useful life (RUL) and the formulation of optimal maintenance policies for aero-engines are critical to mitigating unplanned downtime caused by failures. The deep reinforcement learning (DRL) performs well in addressing the real-time RUL prediction and maintenance decision-making of complex mechanical systems. However, the existing prognostic and maintenance (PaM) methods that rely solely on DRL agents present an insufficient understanding for degraded data when facing complex systems, leading to unreliable maintenance policies and high economic cost. To address these issues, a hybrid framework consisting of the data augmentation model in the front-end and the DRL agent in the back-end is proposed for the PaM of aero-engines. The autoregressive distributed lag (ARDL) model is used for exploring the coupling relationship between health degradation and response degradation, and separating coupling signals helps to quantify the impact of different inducements on equipment degradation. The reliability is derived based on the Wiener process to consider the randomness in the degradation process. The decoupling parameters and reliability are combined to construct the environment state required for the back-end DRL agent training. The generated DRL hybrid framework enhanced by knowledge features is applied to optimize the prognosis and maintenance policies of mechanical equipment considering economic dependence. The C-MAPSS simulation dataset provided by NASA serves as a schematic maintenance case, and the results show that the proposed framework decreases maintenance cost and meets safety requirements for engine PaM tasks.