Safety Factor Profile Control in EAST via Reinforcement-Learning-Based Model Predictive Control

A tokamak is a toroidal device that utilizes helical magnetic fields to confine a superheated plasma. The spatial distribution of the pitch of the magnetic field, referred to as the safety factor profile, is linked to the stability and performance of the confined plasma. Thus, the capability to control the safety factor profile is essential for realizing advanced operation scenarios in tokamaks. This work proposes a Reinforcement Learning-based Model Predictive Control (RLMPC) approach for the enhanced regulation of the safety factor profile in the Experimental Advanced Superconducting Tokamak (EAST). By estimating in real time the uncertain parameters of the plasma model, the RLMPC method aims to achieve more accurate and robust tracking of the target profile. Besides learning the parameterized control-oriented response model, which serves as a linear constraint in the MPC problem, the reinforcement learning-based algorithm also learns the weight associated with the cost function. Swift convergence of the learnable parameters within the RLMPC is facilitated through a second-order Least Squares Temporal Difference Q-learning algorithm. Simulations based on the Control Oriented Transport SIMulator (COTSIM) show that the proposed RLMPC performs better than conventional linear MPC schemes.