Hybrid Two-Level Mixture of Experts Framework Using Machine Learning Components for Day-Ahead Residential Energy Load Forecasting

This paper presents a hybrid two-level Mixture of Experts (MoE) framework for day-ahead residential electricity load forecasting, designed to improve prediction accuracy while meeting the operational and privacy requirements of utility-scale applications. Developed within the Horizon Europe DEDALUS project, the proposed approach addresses the challenge of forecasting highly heterogeneous household energy demand using only anonymised historical electricity consumption data, enabling scalable customer-level forecasting without relying on sensitive household metadata. The proposed methodology combines K-Means clustering with gradient boosting models, employing XGBoost and LightGBM as specialised expert components within a hybrid two-level MoE architecture. By separating the identification of daily consumption regimes from hourly load prediction, the framework captures diverse residential consumption behaviours while maintaining computational efficiency. The forecasting model integrates historical energy consumption, weather forecasts and temporal features to generate day-ahead predictions that support demand response, dynamic tariff schemes and other utility-side energy management services. The framework is validated using real-world smart meter data collected from households participating in the HERON Living Lab. The experimental results demonstrate that the proposed MoE architecture consistently outperforms standalone forecasting models, particularly for households characterised by highly variable and occupancy-driven electricity consumption patterns. The study highlights the potential of hybrid machine learning architectures to deliver accurate, privacy-preserving and scalable residential load forecasting for future smart energy systems and customer-centric energy services.