Optimal Configuration of Transformer–Energy Storage Deeply Integrated System Based on Enhanced Q-Learning with Hybrid Guidance

This paper investigates the multi-objective siting and sizing problem of a transformer–energy storage deeply integrated system (TES-DIS) that serves as a grid-side common interest entity. This study is motivated by the critical role of energy storage systems in generation–grid–load–storage resource allocation and the superior capability of artificial intelligence algorithms in addressing multi-dimensional, multi-constrained optimization challenges. A multi-objective optimization model is first formulated with dual objectives: minimizing voltage deviation levels and comprehensive economic costs. To overcome the limitations of conventional methods in complex power systems—particularly regarding solution quality and convergence speed—an enhanced Q-learning with hybrid guidance algorithm is proposed. The improved algorithm demonstrates strengthened local search capability and accelerated late-stage convergence performance. Validation using a real-world urban power grid in China confirms the method’s effectiveness. Compared to traditional approaches, the proposed solution achieves optimal TES-DIS planning through autonomous learning, demonstrating (1) 70.73% cost reduction and (2) 89.85% faster computational efficiency. These results verify the method’s capability for intelligent, simplified power system planning with superior optimization performance.