Time-varying neural networks based on local attention mechanism for time series forecasting

To address the challenges of long-range dependency capture and local dynamics sensitivity in time series prediction, this study proposes a novel time-varying neural network (LATVN) based on the local attention mechanism. The design of the LATVN focuses on mitigating the gradient vanishing problem and enhancing the model’s sensitivity to the local dynamics in the time series. The model efficiently captures key temporal dependencies by focusing on specific parts of the sequence through local attention blocks (LA blocks). Meanwhile, the time-varying structure allows the model parameters to be dynamically adjusted over time steps, thus better adapting to the non-stationary and cyclical changes in the time series. Extensive experimental evaluations on several standard datasets show that LATVN outperforms existing baseline methods in several evaluation metrics, proving its effectiveness and superiority in time series forecasting tasks. The proposal of LATVN provides new perspectives and solutions in the field of time series forecasting, and it is expected to demonstrate its forecasting potentials in a wider range of application scenarios.