Efficiency of maximum likelihood estimation for a multinomial distribution with known probability sums

For a multinomial distribution, suppose we have prior knowledge about the sum of the probabilities of certain categories. This enables the construction of a submodel within the full (i.e., unrestricted) model. The maximum likelihood estimator (MLE) under such a submodel is generally expected to achieve higher estimation efficiency than the MLE under the full model. However, this article shows that this expectation does not always hold. We derive the asymptotic expansion of the risk of the MLE, with respect to Kullback–Leibler divergence, for both the full model and the “m-aggregation” submodel. The results indicate that the second-order term (the order $$n^{-2}$$ term) of the submodel is larger than that of the full model unless the submodel is based on “solid” prior knowledge. From this theoretical finding, we conjecture that in some cases the submodel may yield a higher risk than the full model. We confirm this conjecture by presenting an explicit example in which the use of the submodel increases the risk.

Authors

Institutions

Publication Details

Journal
Information Geometry
Published
2025-10-18
DOI
https://doi.org/10.1007/s41884-025-00179-y
Primary Topic
Bayesian Methods and Mixture Models
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Efficiency of maximum likelihood estimation for a multinomial distribution with known probability sums

Yo Sheena
Information Geometry
Bayesian Methods and Mixture Models
article

Efficiency of maximum likelihood estimation for a multinomial distribution with known probability sums

Yo Sheena
article en

Abstract

For a multinomial distribution, suppose we have prior knowledge about the sum of the probabilities of certain categories. This enables the construction of a submodel within the full (i.e., unrestricted) model. The maximum likelihood estimator (MLE) under such a submodel is generally expected to achieve higher estimation efficiency than the MLE under the full model. However, this article shows that this expectation does not always hold. We derive the asymptotic expansion of the risk of the MLE, with respect to Kullback–Leibler divergence, for both the full model and the “m-aggregation” submodel. The results indicate that the second-order term (the order $$n^{-2}$$ term) of the submodel is larger than that of the full model unless the submodel is based on “solid” prior knowledge. From this theoretical finding, we conjecture that in some cases the submodel may yield a higher risk than the full model. We confirm this conjecture by presenting an explicit example in which the use of the submodel increases the risk.

Information GeometryVol. 8(2)
Shiga University (JP)
Japan Society for the Promotion of Science
Openalex Percentile: Top 84%
Bayesian Methods and Mixture Models
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.