Prediction of weather and financial time-series data via a Hamiltonian-based filter-projection approach

<h2>Summary</h2> The generalized Langevin equation (GLE), derived from a many-body Hamiltonian by projection, exactly describes the dynamics of coarse-grained variables in complex environments. However, real-world data analysis with the GLE is hampered by transient and seasonal components as well as time-discretization effects. Machine learning (ML) techniques work but are computationally expensive and difficult to interpret. We show that convolution filtering decomposes time-series data into fast, transient, and seasonal components, which can be analyzed separately by projection techniques. We introduce methods to extract GLE parameters from highly discretized time-series data and forecast future data, including environmental stochasticity. For daily resolved weather data, our analysis reveals non-Markovian memory lasting a few days. Our prediction accuracy is comparable to commercial and ML long short-term memory (LSTM) methods at a reduced computational cost by a factor of <mml:math><mml:mrow><mml:msup><mml:mn>10</mml:mn><mml:mn>2</mml:mn></mml:msup><mml:mo>−</mml:mo><mml:msup><mml:mn>10</mml:mn><mml:mn>3</mml:mn></mml:msup></mml:mrow></mml:math>. For financial data, short-ranged memory confirms Markovian dynamics, consistent with the efficient-market hypothesis; consequently, models simpler than the GLE are sufficient. Thus, our GLE framework is an efficient and interpretable method for analyzing and predicting complex time-series data.