Dynamical equations of mutual information and transfer entropy in discrete time

The relationship between mutual information rate and transfer entropy—two central concepts in information dynamics—has been examined primarily in bipartite systems. This paper extends the analysis to general systems by introducing a unified information-theoretic framework that derives exact dynamical equations for entropy, mutual information, transfer entropy, and reversed transfer entropy in discrete time. Unlike approaches based on the master equation or the Fokker–Planck equation, the present formulation treats information measures as dynamical state variables governed by intrinsic equations with control variables, providing a purely information-theoretic description. In the zero-control regime of this framework, all state variables are shown to evolve linearly in time, establishing that nonzero control variables induce curvature in information evolution. Near this limit, nontrivial dynamics arise on ring and Bethe lattice topologies under directional constraints on information flow. This analysis is supported by simulations of a biological model of gene regulatory networks. Analysis of a bivariate Markov chain further establishes that the control variables are directly linked to the stochastic nature of the system. Furthermore, an erase operation analogous to Landauer’s principle is examined within this formulation, revealing that nonzero reversed transfer entropy plays a crucial role in information erasure. In conclusion, this framework offers a general foundation for studying information dynamics in complex systems, with potential applications ranging from information thermodynamics and neural networks to biological systems and critical phenomena.