Recurrent neural network trained with the extended Kalman filter to forecast the geomagnetic secular variation for IGRF-14

Abstract This study proposes a neural network approach for predicting the geomagnetic secular variation (SV) to improve the accuracy and efficiency of short-term geomagnetic forecasts. The International Geomagnetic Reference Field (IGRF), updated every five years, provides a standardized representation of Earth’s magnetic field, including a 5-year linear prediction of SV. Recent forecasting methods, which are reliant on computationally intensive geodynamo simulations, often struggle to capture sudden changes due to nonlinearity, such as the geomagnetic jerk. We have developed a novel recurrent neural network (RNN) framework trained using the extended Kalman filter (EKF), termed the EKF-RNN, to address these challenges. Unlike conventional backpropagation methods, the EKF dynamically updates the RNN weights by incorporating error covariance from training data, effectively mitigating overfitting while preserving computational efficiency. The EKF-RNN model is validated through hindcast experiments for epochs 2004.87 to 2014.62, utilizing geomagnetic field snapshots derived from satellite and ground magnetic observations. The results exhibit forecast errors below 85 nT for 5-year predictions, outperforming known data assimilation methods such as 4dEnVar. Additionally, the EKF-RNN method provides forecast error covariance matrices, offering enhanced interpretability and robustness compared to earlier neural network models. This research underscores the potential of EKF-RNN for reliable geomagnetic SV predictions, contributing to the accuracy of the 14th-generation IGRF and advancing data-driven approaches in geomagnetic field modeling. Graphical Abstract