Physics-informed Feature Engineering in Machine Learning-Based Forecasting of Variable Renewable Energy

The United States aims to achieve a 100% clean energy grid by 2035, placing Variable Renewable Energy (VRE) sources such as solar and wind at the forefront of this transition. In 2023, VRE accounted for 14.1% of total energy generation in the US, and it is projected that by 2045, VRE will account for 40-70% of total energy generation. This rapid increase underscores the growing importance of VRE in the nation's energy landscape. However, integrating VRE into the grid presents significant challenges due to its inherent variability and unpredictability. Accurate forecasting of VRE generation is crucial for effectively managing these variable resources. This dissertation addresses the urgent need for accurate, transparent, and robust forecasting models for VRE generation. Traditional Numerical Weather Prediction (NWP) models, while accurate, are time-consuming and computationally expensive, limiting their real-time applicability. In contrast, machine learning (ML) models offer faster predictions and computational efficiency, making them crucial for the changing energy landscape. However, ML models often lack transparency and rely on extensive datasets, which can be inconsistent due to factors like curtailment and outages. To overcome these challenges, this research integrates physics-informed feature engineering into state-of-the-art ML models, hypothesizing that this integration will enhance day-ahead VRE forecasting accuracy by over 10% and reduce systematic biases in forecasting. Additionally, it examines the impact of curtailed historical generation data and varying weather forecasting accuracy impact on ML-based VRE forecasting models. The dissertation evaluates six advanced ML models: Random Forest (RF), Extreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LGBM), Long- Short Term Memory (LSTM), Gated Recurrent Unit (GRU), and Convolution Neural Network (CNN). By incorporating physics-informed features such as clear sky irradiation models (Ineichen, Solis), cell temperature model, Bulk Richardson Number (RiB), wind gradient, sine-cosine transformations, polynomial features, and Discrete Wavelet Transformations (DWT) of wind speed, the models are tested on data from 20 solar farms and 20 wind farms across California. Results demonstrate that integrating physics-informed features significantly improves forecasting accuracy, with an over 10% increase across all case study sites. The study also reveals that curtailed generation data impacts solar power forecasting accuracy by 3-5%, while accurate weather forecasting can lead to over 98% day-ahead VRE forecasting accuracy. Tree-based models like RF excel in day-ahead solar power forecasting, whereas neural network models like LSTM are most effective for day-ahead wind power forecasting.