Real-time building energy storage scheduling under electrical load uncertainty: A Markov decision process approach with comprehensive analysis of different pricing policies

This study presents a novel real-time Markov decision process (MDP) model for scheduling battery systems in buildings with uncertain electrical loads. Unlike conventional methods, the model integrates quantile Fourier regression for more accurate load fitting and better management of load uncertainty. The result is a large-scale optimization problem with approximately one million decision variables and constraints significantly increasing complexity over previous models. By reformulating the problem as a linear program, the model becomes more computationally efficient and suitable for real-time applications. Using the commercial solver to address the problem enables rapid identification of optimal solutions, making the model practical for dynamic energy environments. A key innovation is the code's adaptability to various pricing policies and demand scenarios, including peak shaving and time-of-use pricing, which are increasingly important in modern energy markets. The model was validated using a full year of historical demand data from a school, providing a comprehensive test of its performance. Results show that the MDP model can dynamically adapt to load uncertainty with different performance efficiencies depending on the pricing policy. The proposed flexible data-driven approach offers significant advantages over static methods, particularly in environments where load uncertainty poses a critical challenge. The performance of the MDP is also compared with machine learning and neural Markov models. • A model for real-time energy storage scheduling under load uncertainty is present. • A novel integrated for the MDP model with Quantile Fourier regression is proposed. • The model is validated model using historical data and tested under various tariffs. • A 90% efficiency in key pricing policies is achieved by the model. • The proposed model has a dynamic pricing for future energy market alignment.