Quasi-differentiation and its applications to noisy time series data from complex systems

The analysis of time series data from a complex system is challenging, because apart from its multiple stable states arising as low-dimensional manifolds, contribution from the rest of the many variables manifest themselves as state-dependent noise. To identify and characterize these stable states, and detect transitions between them, we need to construct slowly varying order parameters from the noisy time series. In this paper, we propose a model-free method to extract the slowly varying parts of noisy time series data, by taking the difference between the integrated information in a sliding pair of adjoining time windows. Because this differential information has the structure of a derivative, we call it a quasi-derivative and the method quasi-differentiation. We tested this method on some simple examples, before applying it successfully to identify the Oct 2008 Lehman Brothers and Mar 2020 COVID-19 market crashes in the daily returns of the Dow Jones Industrial Average (DJIA) index from 2003 to 2023. We then describe how we can approximate the slowly varying part of the noisy time series, by re-integrating the quasi-derivative obtained. After testing this method of integrated quasi-differentiation on a few other examples, we applied it successfully to extract the slowly varying mean and variance of the DJIA. Finally, we discuss how the method of integrated quasi-differentiation can be used to obtain point estimates of the Hurst exponent and linear cross correlation. Although we have illustrated the above methods on stock market data, we believe they can be applied to a large variety of quantities in many other complex systems.