A hybrid model predictive control-deep reinforcement learning algorithm with application to plug-in electric vehicles smart charging

This paper focuses on a novel use of deep reinforcement learning (RL) to optimally tune in real-time a model predictive control (MPC) smart charging algorithm for plug-in electric vehicles (PEVs). The coefficients of the terminal cost function of the MPC algorithm are updated online by a neural network, which is trained offline to maximize the control performance (linked to the satisfaction of the users’ charging preferences and the tracking of a power reference profile, at PEV fleet level). This approach is different and more flexible compared to most of the other approaches in the literature, which instead use deep RL to fix offline the MPC parametrization. The proposed method allows one to select a shorter MPC control window (compared to standard MPC) and/or a shorter sampling time, while improving the control performance. Simulations are presented to validate the approach: the proposed MPC-RL controller improves control performance by an average of 4.3 % compared to classic MPC, while having a lower computing time. • We present a novel smart charging algorithm based on combined model predictive control (MPC) and deep reinforcement learning (RL). • We use deep RL to train a neural network that selects in real time the weights of the MPC terminal cost function. • The optimal selection of the terminal cost weights boosts the performance of the MPC smart charging algorithm. • Simulations show that the proposed approach improves the smart charging performance and can also be used to reduce the MPC computational time.