Short-Term Wind Power Prediction Based on Wind2vec-BERT Model

In the era of new energy development, the requirements for all aspects of short-term wind power forecasting tasks are increasing day by day. However, the power condition of wind farms is naturally stochastic and variable as it is affected by multiple factors. Current neural network approaches focus only on the propagation of unidirectional attention and ignore the interaction of input variables. To further improve the accuracy of wind power prediction, this paper explores the application of the Bidirectional Encoder Representations from Transformers (BERT) algorithm in wind power prediction. At the same time, GARCH series models are used for analysis and optimization after the prediction results are obtained to address the challenges posed by the inherent variability of wind. Meanwhile, Wind2vec, a new variable embedding method for wind power forecasting tasks, is proposed which can more efficiently fit the relationship between time series forecasting variables. The parameters are subsequently fine-tuned for the backbone layer of the BERT using the Adaptive Computation Time (ACT) method to make it more adaptive to the inputs of the power sequences of the power system. By BERT's bidirectional attention mechanism and transformer architecture, and refining it for the input layer, we aim to enhance the accuracy of wind power forecasts by capturing nuanced temporal dependencies within historical wind data. Using China Southern Power Grid real datasets demonstrates the effectiveness and correctness of the BERT-GARCH-M-based model in outperforming traditional forecasting methods. This research not only shows the adaptability of BERT to wind power prediction but also contributes to advancing the precision and reliability of renewable energy forecasts, paving the way for more sustainable energy utilization in the evolving landscape of new energy paradigms.