Hybrid AC/DC Transmission Grid Planning Based on Improved Multi-Step Backtracking Reinforcement Learning

Hybrid AC/DC transmission expansion planning must balance investment cost, supply reliability and AC/DC stability, which challenges conventional mathematical programming and heuristic methods. This paper proposes a multi-objective planning framework based on an improved multi-step backtracking α-Q(λ) reinforcement learning algorithm with eligibility traces and an adaptive learning factor. A tri-objective model minimises annual economic cost, expected power shortage and a comprehensive electrical index that combines electrical betweenness, commutation-failure margin and effective short-circuit ratio. The mixed-integer planning problem is reformulated as an interactive learning process, where the state encodes candidate line construction decisions, the action builds or cancels lines, and the eligibility-trace matrix is used to quantify line importance. Case studies on the Garver-6 system, the IEEE 24-bus reliability test system and a 500 kV regional hybrid AC/DC grid show that, compared with classical Q-learning, the proposed method yields lower annual cost, reduced expected power shortage and improved AC/DC stability; in the 500 kV system, the expected annual power shortage is reduced from 70,810 MWh to 28,320 MWh.