Investigation of Transformer-based models performance on solar irradiance forecast: Empirical results and analysis

Solar irradiance forecasting is of paramount importance in the efficient management of renewable energy systems. Transformer-based models have garnered substantial interest in this field due to their exceptional scalability and performance. However, the influence of various factors, such as temporal scales, regional disparities, and weather conditions, on their forecasting capabilities remains an underexplored area. To elucidate the inherent strengths of Transformers and their variants in irradiance forecasting, an exhaustive assessment of the Transformer and its derivatives is proposed, employing the most recent models. Nine models, including the Transformer and its eight enhanced versions, are compared across short-term, mid-term, and long-term tasks using data from two public databases in three distinct regions. The evaluation encompasses various performance metrics and diverse weather conditions, assessing both overall performance and the effects of parameters like time lag, forecast horizon, training dataset size, and sampling rate. The results demonstrate that Transformer-based models consistently exhibit strong performance on irradiance forecast tasks, particularly on long-term forecasts. Among Transformer variants, the performance of the best and worst approaches can differ by approximately 30.5%–70%. Moreover, strategic simplification of the Transformer's attention mechanism computational complexity not only preserves its long-term forecasting advantages but also significantly improves its capability in short-term irradiance prediction tasks, an area where conventional Transformer architectures typically underperform.