Augmenting Neural Networks With Time‐Varying Weights

ABSTRACT In the macroeconomic forecasting community, there is increasing interest in machine learning methods that can extract nonlinear predictive content from large datasets with a high number of predictors. Meanwhile, time‐varying parameter (TVP) models are known to flexibly model time series by allowing regression coefficients to vary over time. This paper generalizes neural networks to allow for time variation of the weights of the final layer. The variance components of the time‐varying weights are estimated alongside the fixed network weights via an EM algorithm. The result is the time‐varying neural network (TVNN), a fully supervised, nonlinear model, which combines the desirable properties of classical econometric approaches with the predictive capacity of neural networks. The TVNN model yields improved forecasts over similarly tuned feedforward neural networks with fixed weights, recurrent network architectures, and benchmark autoregressive models on time series from the popular FRED‐MD database.