Deposit and withdraw: Reinforcement learning‐based incentive design for shared energy storage

Abstract Many residential prosumers exhibit a high price tolerance for household electricity bills and a low response to price incentives. This is because household electricity bills are not inherently high, and the potential for saving electricity bills through participation in conventional shared energy storage (SES) is limited, which diminishes their motivation to actively engage in SES. Additionally, existing SES models often require prosumers to take additional actions, such as optimising rental capacity and bidding prices, which happen to be capabilities that typical household prosumers do not possess. To incentivise these high‐price‐tolerance residential prosumers to participate in SES, a novel SES aggregation framework is proposed, which does not require prosumers to take additional actions and allows them to maintain existing energy storage patterns. Compared to the conventional long‐term operation of SES, the proposed framework introduces an additional short‐term construction step during which the energy service provider (ESP) acquires control of the energy storage systems (ESS) and offers electricity deposit and withdrawal services (DWS) with dynamic coefficients, enabling prosumers to withdraw more electricity than they deposit without additional actions. Additionally, a matching mechanism is proposed to align prosumers’ electricity consumption behaviours with ESP optimisation strategies. Finally, the dynamic coefficients in the DWS and SES trading strategies are jointly optimised using a modified deep reinforcement learning algorithm. Combining neighbouring experience pool replay modifies the twin delay deep deterministic policy gradient (CNEPR‐TD3), which introduces a multilabel neighbouring experience replay mechanism to improve learning efficiency and convergence stability. Simulation studies based on one‐year real‐world data validated the proposed approach. Ablation experiments showed that the inclusion of dynamic DWS and the matching mechanism increased the overall SES profit by 42.87%, confirming the effectiveness and economic value of the proposed framework.