Common Information Dimension

Quantifying the common information between random variables is a fundamental problem with a long history in information theory. Traditionally, common information is measured in number of bits and thus such measures are mostly informative when the common information is finite. However, the common information between continuous variables can be infinite; in such cases, a real-valued random vector <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">W</i> may be needed to represent the common information, and to be used for instance for distributed simulation. In this paper, we propose the concept of Common Information Dimension (CID) and three variants. We compute the common information dimension for jointly Gaussian random vectors in a closed form. Moreover, we analytically prove, under two different formulations, that the growth rate of common information in the nearly infinite regime is determined by the common information dimension, for the case of two Gaussian vectors.