Geometry of Statistical Manifolds

A statistical manifold <i>M</i> can be defined as a Riemannian manifold each of whose points is a probability distribution on the same support. In fact, statistical manifolds possess a richer geometric structure beyond the Fisher information metric defined on the tangent bundle TM. Recognizing that points in <i>M</i> are distributions and not just generic points in a manifold, TM can be extended to a Hilbert bundle HM. This extension proves fundamental when we generalize the classical notion of a point estimate-a single point in <i>M</i>-to a function on <i>M</i> that characterizes the relationship between observed data and each distribution in <i>M</i>. The log likelihood and score functions are important examples of generalized estimators. In terms of a parameterization θ:M→Θ&sub;Rk, θ^ is a distribution on Θ while its generalization gθ^=θ^-Eθ^ as an <i>estimate</i> is a function over Θ that indicates inconsistency between the model and data. As an <i>estimator</i>, gθ^ is a distribution of functions. Geometric properties of these functions describe statistical properties of gθ^. In particular, the expected slopes of gθ^ are used to define Λ(gθ^), the Λ-information of gθ^. The Fisher information <i>I</i> is an upper bound for the Λ-information: for all <i>g</i>, Λ(g)≤I. We demonstrate the utility of this geometric perspective using the two-sample problem.