Optimal Control of District Cooling System With Thermal Energy Storage Based on Twin Delayed Deep Deterministic Policy Gradient

China has been rapidly expanding its wind and solar power generation capacity to support carbon neutrality in the power sector. However, the intermittent and unpredictable nature of these energy sources has made it harder to maintain a real-time supply-demand balance. One key strategy to address this is demand response, where flexible loads are adjusted to align with renewable energy generation through time-of-use or real-time pricing. Air conditioning, which can account for over 50% of peak summer electricity demand in some cities, is a prime target for demand response due to buildings' thermal inertia. This paper focuses on optimizing demand response control for large-scale district cooling systems under real-time pricing. We propose a reinforcement learning method based on a twin delayed policy gradient to optimize ice storage systems, managing uncertainties from both cooling demand and electricity prices. First, we model the problem as a Markov decision process, then adopt a model- free twin delayed deep deterministic policy gradient algorithm to improve training efficiency and convergence. Simulation experiments validate the effectiveness of this control approach.