Three Approaches to Approximating the Fisher Information Number for Gaussian Mixture Densities

The Fisher information number (FIN) has previously been proposed as a regularizer to fit a probability density function to a set of constraints. Especially for mixture densities, this is not straightforward and often a reformulation based on square root densities is used. As it is generally much harder to derive the square root of a mixture than squaring it, this only allows for constraints that can be expressed through the root density's parameters. An important case not covered by this are constraints on individual components of a mixture. This paper proposes three methods to approximate the FIN of mixture models: Gauss-Hermite quadrature, polynomial approximation of the square root function, and direct approximation of the square root density of a pdf. This allows using the FIN for smooth density estimation in situations existing methods cannot handle. The three methods are applied to the problem of kernel density estimation with Gaussian kernels and the results are compared.