A systematic study of Echo State Networks topologies for chaotic time series prediction

In the last twenty years, Echo State Networks have become a prominent method for the prediction of time series with a large variety of proposed topologies of connections. These topologies inside the ESN are typically shown to be advantageous for a certain prediction task in terms of reducing the computational complexity and increasing the explainability regarding the selection of weights. Still, a thorough comparison of these different topologies is missing in terms of their formal description and their performance as well as characteristics when applied to different prediction tasks. In this paper, we study 16 topologies for the task of time series prediction in the context of chaotic dynamics. We restrict our focus to those since they are considered among the most challenging to predict, while still being of interest for applications and being representative for other time series. We categorize the selected topologies into intralevel connections, concurrent and sequential reservoirs as well as complex topologies and implemented all of them. We parametrized all according to their original publications but also ran additional tests regarding suitable parametrization. All topologies are evaluated on the eight time series, in auto-regressive as well as single-step predictions. Our results emphasize on the benefit of choosing a suitable topology, reducing the error by up to 94.87% compared to the baseline, the conventional Echo State Network. We also compared the 16 topologies to the widely used Gated Recurrent Unit (GRU) network and observe the structured reservoir yielding a decreased error of up to 99.99% compared to the GRU.