Information geometry of Gaussian processes and its applications to transfer learning

Gaussian Processes (GPs) are widely used in machine learning for modeling complex functions with uncertainty quantification. Yet their infinite-dimensional nature creates challenges for building a principled geometric framework, especially in defining the Kullback–Leibler (KL) divergence. We propose a novel information-geometric framework that applies to a general class of GPs, beyond posterior models or those with shared priors. Our approach introduces an averaged KL divergence that avoids divergence to infinity and is independent of specific input choices. To support this formulation, we extend the model class to Integrated Gaussian Processes (IGPs), which relax the marginal consistency requirement. The resulting IGP space forms a dually flat manifold, within which the GP space appears as an m-flat submanifold. As an application, we demonstrate transfer learning by projecting a target GP onto the convex hull of source GPs via an e-projection. To overcome the difficulty of infinite-dimensional parameters, we employ a geometric optimization algorithm based on the generalized Pythagorean theorem. This establishes a mathematically principled and computationally feasible foundation for transfer learning, while also providing a versatile toolset for geometrically grounded applications of GPs.