Recurrent Deterministic Actor-Critic Algorithm for Real-Time Demand Management in Destination Charging Stations

The increasing calls for the participation of prosumers and Distributed Energy Resources (DERs) have led to a three-fold shift in the operation of distribution networks, i.e., the evolution of Distribution System Operator (DSO), Decentralized Architecture for local-level markets, and real-time control algorithms. This article proposes a real-time recurrent reinforcement learning scheduling algorithm for electric vehicles (EVs) charging at a destination charging station (DCS). Most EVs at these DCSs remain plugged in for longer than the required charging time, thereby offering inherent scheduling flexibility. The proposed algorithm would enable DCS to manage the charging and waiting queues, exploit EVs’ flexibility, manage their charging demands, and provide multiple grid services to the DSO. This algorithm relies only on the state of charge (SOC) of the EV when it arrives and its expected departure time to determine the rate of charging for the duration of its stay. Additionally, we propose a transparent charging price scheme for EV owner’s time-based flexibility. The DCS utilizes a recurrent deterministic actor-critic reinforcement learning method to determine the rate of charging (current drawn) of all plugged-in ports in the station to fulfill the individual EV charging demand before departure. It schedules the overall charging demand to offer flexibility services to the DSO at an incentive. The article highlights a realistic charging station environment to meet EV charging needs satisfactorily and compares the proposed algorithm to other Reinforcement Learning methods to show its efficacy.