AI-Based Forecasting in Renewable-Rich Microgrids: Challenges and Comparative Insights

Amid the accelerating global transition to renewable energy, accurate forecasting has become the cornerstone for unlocking the full potential of solar and wind power in modern power grids, especially in regions with high resource variability. This study begins with a review of forecasting challenges in microgrids located in developing areas where issues related to data sparsity, model limitations, environmental variability, and operational limitations are prevalent. To address some of these challenges, we benchmarked twelve machine learning (ML) and deep learning (DL) models using high-resolution hourly data from South Africa’s national grid data, covering Energy Demand, Photovoltaic (PV), and Wind generation over five years. Our results show that Bidirectional-Long Short-term Memory (Bi-LSTM) achieved the best demand and PV forecasting performance with a Root Mean Square Error (RMSE) / R² of 303.95, Megawatt MW/ 0.9877 and 48.04 MW / 0.996, while XGBoost delivered a competitive accuracy for PV (RMSE: 49.74 MW, R²: 0.996) and Wind (RMSE: 119.77 MW, R²: 0.964). Classical ML models outperformed most DL architectures, including Transformer and Convolutional Neural Network (CNN)-LSTM, which underperformed despite their complexity. Statistical testing via Friedman (χ² = 10,858.39, p < 0.00001) and Nemenyi post-hoc tests confirmed significant differences in model performance. The study demonstrates that, with effective feature engineering, classical ML models can rival deep learning counterparts in forecasting accuracy. This finding provides actionable insights for utilities and policymakers by highlighting robust, low-latency models suitable for practical deployment in operational power systems.