Sensor-Generated In Situ Data Management for Smart Grids: Dynamic Optimization Driven by Double Deep Q-Network with Prioritized Experience Replay

According to forecast data from the State Grid Corporation of China, the number of terminal devices connected to the power grid is expected to reach the scale of 2 billion within the next five years. With the continuous growth in the number and variety of terminal devices in the smart grid, traditional cloud-edge-end architecture will face the increasing issue of response latency. In this context, in situ computing, as a paradigm for local or near-source data processing within cloud-edge-end architecture, is gradually becoming a key technological pathway in industrial systems. The in situ server system, by deploying servers near terminals, enables the near-data-source processing of terminal-generated in situ data, representing an important implementation of in situ computing. To enhance the processing efficiency and response capability of in situ data in smart grid scenarios, this study designs an in situ data processing mechanism and an access demand management framework. Due to the heterogeneity of in situ server performance, there are variations in response capabilities for access demands across rounds. This study introduces a double deep Q-network with prioritized experience replay to assist in response decision-making. Simulation experiments show that the proposed method reduces waiting latency and response latency by an average of 67.69% and 68.77%, respectively, compared to traditional algorithms and other reinforcement learning algorithms, verifying its effectiveness in in situ data management. This scheme can also be widely applied to in situ computing scenarios with low-latency data management, such as smart cities and industrial IoT.