New Formulas of Feedback Capacity for AGN Channels with Memory: A Time-Domain Sufficient Statistic Approach

Recently, several papers identified technical issues related to equivalent time-domain and frequency-domain "characterization of the <i>n</i>-block or transmission" feedback capacity formula and its asymptotic limit, the feedback capacity, of additive Gaussian noise (AGN) channels, first introduce by Cover and Pombra in 1989 (IEEE Transactions on Information Theory). The main objective of this paper is to derive new results on the Cover and Pombra characterization of the <i>n</i>-block feedback capacity formula, and to clarify the main points of confusion regarding the time-domain results that appeared in the literature. The first part of this paper derives new equivalent time-domain sequential characterizations of feedback capacity of AGN channels driven by non-stationary and non-ergodic Gaussian noise. It is shown that the optimal channel input processes of the new equivalent sequential characterizations are expressed as functionals of a <i>sufficient statistic and a Gaussian orthogonal innovations process.</i> Further, the Cover and Pombra <i>n</i>-block capacity formula is expressed as a functional of two generalized matrix difference Riccati equations (DREs) of the filtering theory of Gaussian systems, contrary to results that appeared in the literature and involve only one DRE. It is clarified that prior literature deals with a simpler problem that presupposes the state of the noise is known to the encoder and the decoder. In the second part of this paper, the existence of the asymptotic limit of the <i>n</i>-block feedback capacity formula is shown to be equivalent to the convergence properties of solutions of the two generalized DREs. Further, necessary and or sufficient conditions are identified for the existence of asymptotic limits, for stable and unstable Gaussian noise, when the optimal input distributions are asymptotically time-invariant but not necessarily stationary. This paper contains an in-depth analysis, with various examples, and identifies the technical conditions on the feedback code and state space noise realization, so that the time-domain capacity formulas that appeared in the literature, for AGN channels with stationary noises, are indeed correct.