Optimizing Grid Services: A Deep Deterministic Policy Gradient Approach for Demand-Side Resource Aggregation

The new types of demand-side controllable resources in recent years, such as smart home appliances, portable power stations, and electric vehicles, have introduced newfound opportunities for smart grid control. Aggregating a large volume of these resources can provide a substantial amount of ancillary services for the grid, such as voltage and frequency control. Nevertheless, the integration of such a large and highly distributed amount of controllable resources for grid services has posed significant challenges for grid operators as well as local energy community aggregators, primarily due to the uncertainties, load characteristics, computational complexity, and scalability. This paper aims to devise an efficient distributed aggregation system targeting the optimal control and integration of diverse controllable resources for power system services. This is achieved through a model-free deep reinforcement learning-based approach, which not only maximizes the total profits of the distributed resource owners to incentivize participation but also improves the control performance. The efficacy of the proposed system is validated through case studies utilizing real-world data.