Prototype Federated Reinforcement Learning for Voltage Regulation in Distribution Systems With Physics-Aware Spatial-Temporal Graph Perception

Online voltage regulation in active distribution systems faces challenges stemming from privacy protection concerns and uncertainties introduced by renewable energy sources. To address these issues, a novel spatial-temporal transformer-based prototype federated reinforcement learning (STT-PFRL) model is proposed to mitigate voltage deviations while ensuring data privacy. Specifically, STT-PFRL operating within a decentralized framework trains the model by transmitting local prototype information between a central data server and local agents, avoiding raw data privacy leakage. Besides, a novel physics-aware spatial-temporal transformer network is proposed to improve the voltage regulation policy learning stability against uncertainties by embedding the spatial-temporal graphical physics information into the data aggregation process. Furthermore, the prototype learning-based federated soft actor-critic (ProtoFedSAC) algorithm incorporates a prototype layer to utilize diverse feature representations, thereby enhancing the model's ability to handle heterogeneity in environmental data. Simulation results on 33- and 118-node distribution systems demonstrate the superior effectiveness and efficiency of STT-PFRL in voltage regulation.