Achieving Optimal Demand-Side Integrated Energy Management via Hierarchical End-to-End Learning with Multisource NWP Data

Accurate modeling of renewable energy and load demand uncertainties, along with achieving optimal energy scheduling, are two critical components in the daily energy management of integrated energy systems (IES). This paper constructs an end-to-end pipeline to address the optimal energy management of IES. Specifically, the framework jointly optimizes upstream renewable energy forecasting, load forecasting, and downstream optimal energy scheduling through a seamless pipeline to enhance operational benefits. First, multisource numerical weather prediction (NWP) data are leveraged to construct sister photovoltaic (PV) and load forecasting models. Two sister models from different NWP sources are combined through a learnable linear aggregation layer. A differentiable optimization layer is designed to bridge the forecasting and scheduling stages. In this layer, the sister forecasts derived from heterogeneous data sources are translated into operational costs within the downstream energy scheduling model. These costs are then back-propagated to simultaneously refine both the stacking weights of the forecasting ensemble and the parameters of the individual forecasting models, enabling dual-layer optimization. Numerical experiments conducted on a residential-commercial IES synthesized from real-world electricity load and PV generation data in Anchorage, Alaska, demonstrate that the proposed framework achieves a cost reduction of 4.03% compared to conventional decoupled predict-then-optimize pipelines.