Natural Language Processing Using Reservoir Computers: Network-Theoretic Analysis and Information Flow

Reservoir computing (RC) is a highly efficient time-series processing neural network model that can be applied to a wide range of real-world applications, including natural language processing. This study offers physical perspectives on how RC processes natural language, focusing on network structure and information flow. The character memorization task on Shakespeare’s works (i.e., a 9,044,939-character database) revealed that memory retention largely depends on the network’s structural properties. However, structural property measures such as clustering coefficient, shortest path length, and degree fail to capture the prediction mechanism. Predicting the following character mainly relies on optimal effective memory depth, revealed by Layer-wise relevance propagation analysis. This information-centered view, rather than apparent network topologies, may strongly support the notion of a physical model-based understanding of Large Language Models.