Anomaly Detection of Thermal System Using CAE-DDQN Model

Abstract In the development of the international nuclear power industry, nuclear energy safety has always been an unavoidable major issue, and nuclear energy safety concerns global energy development, public safety and national security. In the process of variable operating conditions of the thermal engineering system, incorrect alarm information may appear in the main control room due to the fluctuations of the system operation, which hinders the judgment of the actual operating state of the thermal engineering system. Although some research work has been carried out in the field of thermal system condition recognition in the world, the relevant research often adopts unsupervised machine learning model or supervised neural network model, and the application of deep reinforcement learning method is rarely studied. The deep reinforcement learning model can interact in real time in the environment, and multiple neural network models inside the deep reinforcement learning model can be trained through the guidance of the designed reward function to output different result values that can better complete the task based on different data. This method often has strong dynamic programming characteristics. It has strong theoretical feasibility in dynamic adjustment of spatial coding of thermal system data. Therefore, by combining the Convolutional Auto-Encode (CAE) method in the auto-encoder and the Double Deep Q-learning (DDQN) method in the deep reinforcement learning, this research constructs an intelligent model for the recognition of abnormal working conditions of thermal systems under time series. This research takes the simulation data of Qinshan Nuclear Power Plant under varying working conditions as the data sources, and compiles the spatial encoding of the time series data through the Convolutional Auto-Encode (CAE) model, which is pre-trained. Take the constructed spatial coding as the state value of the Double Deep Q-learning (DDQN) model, and guide the Double Deep Q-learning (DDQN) model to adjust the state of spatial coding through appropriate reward value. Finally, the ideal test results are obtained. By comparing with traditional anomaly detection methods, it is proved that CAE-DDQN model has a very ideal condition recognition effect, and the recognition effect of difficult to identify data and very early accident data is much stronger than that of traditional machine learning or deep learning methods. At the same time, it is proved that the deep reinforcement learning method based on the principle of Q function is feasible in the condition recognition of thermal engineering system.