In-situ data storage management for smart grid: A dynamic optimization approach based on Categorical Deep Q-Networks with Experience Replay

The exponential proliferation of intelligent terminal devices in smart grids has triggered a massive surge in heterogeneous data, presenting a critical bottleneck for efficient in-situ storage management and real-time processing. Particularly in industrial systems, in-situ computing, as a mode of data processing performed locally or near the data source, has become one of the key technologies for improving data processing efficiency and responsiveness. In the smart grid environment, in-situ server systems deploy server clusters near terminal data sources to achieve efficient processing, storage, and management of in-situ data. To optimize the management of in-situ terminal data in smart grids, this paper proposes a rule-guided classification method, combined with a distributed task queue architecture for organizing and managing storage tasks. Additionally, the paper introduces the Categorical Deep Q-Network with Experience Replay algorithm to further optimize the response latency of storage tasks, enhancing the learning efficiency and accuracy of response decisions. Simulation results show that, compared to traditional algorithms and other reinforcement learning algorithms, the proposed method outperforms in multiple key metrics: average waiting latency is reduced by 54.27%, average unfinished tasks decrease by 69.53%, and average completion latency is lowered by 63.15%. This method has broad application prospects in in-situ computing scenarios such as smart grids and industrial IoT.