Using recurrence microstates to improve learning of multi-layer perceptrons

Abstract Artificial Neural Networks (ANNs), particularly the Multi-Layer Perceptron (MLP), have become popular for modeling complex functions and solving nonlinear problems. Despite their success, the difficulty in interpreting their internal processes hinders the understanding of many physicists of how hyperparameters, input data, and training techniques influence convergence and overall model performance. In this study, we propose the use of recurrence analysis as an additional tool to examine the dynamic behavior of MLP networks during the learning process. Here, the learning process refers to the supervised classification of different dynamical regimes, where each class corresponds to a distinct parameter value of the underlying system. We apply the methodology to several well-known dynamical systems: the generalized Bernoulli shift, logistic map, the Lorenz attractor, and long memory processes known as colored noise. The results indicate that the microstates quantification changes the way the network interprets the data, varying the weight of the connection between layers and forming patterns on connection weight matrices, especially for larger microstate sizes. Furthermore, we observe that the use of recurrence microstates enables the identification of overfitting, providing a more physically interpretable understanding of the network’s internal dynamics. In summary, recurrence analysis can be used as a complementary approach for interpreting and enhancing the performance of neural networks, especially in tasks involving complex data and dynamical systems.