Duality symmetry, two entropy functions, and an eigenvalue problem in generalized Gibbs’ theory

Abstract We generalize the convex duality symmetry in Gibbs’ statistical ensemble formulation, between the Gibbs entropy φ V , N ( E ) as a function of mean internal energy E and Massieu’s free entropy Ψ V , N ( β ) as a function of inverse temperature β . The duality in terms of Legendre–Fenchel transform tells us that Gibbs’ thermodynamic entropy is to the law of large numbers (LLN) for arithmetic sample mean values what Shannon’s information entropy is to the LLN for empirical counting frequencies in independent and identically distributed data. Proceeding with the same mathematical logic, we identify the energy of the state { u i } as the conjugate variable to the counting of statistical occurrence { m i } and find a Hamilton–Jacobi equation for the Shannon entropy analogous to an equation of state in thermodynamics. An eigenvalue problem that is reminiscent of certain features in quantum mechanics arises in the entropy theory of statistical counting frequencies of Markov correlated data.