Greedy Empirical Bayes for Density Estimation

Bayesian nonparametric density estimation is challenging in high dimensions, where traditional approaches like the Dirichlet process (DP) mixture rely on computationally intensive Markov chain Monte Carlo or approximations. We introduce the partition posterior bootstrap (PPB), where a flexible prior is constructed by smoothing a DP over a partition of the sample space. Conditioned on this partition, exact posterior sampling reduces to a simple Bayesian bootstrap-like algorithm. Importantly, we select the partition using a greedy empirical Bayes (GEB) algorithm which scales to high dimensions and large sample sizes. GEB greedily maximizes the closed-form marginal likelihood via local split decisions arising naturally from its factorization, yielding an adaptive binary tree partition. Our key theoretical result is that the GEB tree belongs to an admissible family with high probability, splitting all identifiable signals while avoiding over-splitting noise. As a result, the PPB posterior with the GEB tree attains adaptive contraction rates. Our theory holds under weak, verifiable assumptions, which we demonstrate in numerous examples. Empirically, PPB delivers accurate density estimates and uncertainty quantification at a fraction of the computational cost of other approaches, making it a practical off-the-shelf choice for high-dimensional Bayesian density estimation with theoretical guarantees.

Authors

Publication Details

Journal
arXiv (Cornell University)
Published
2026-11-17
DOI
https://doi.org/10.48550/arxiv.2611.22796
Primary Topic
Bayesian Methods and Mixture Models
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
preprint

Greedy Empirical Bayes for Density Estimation

Edwin Fong, Taole Sha, Shibo Yu
arXiv (Cornell University)
Bayesian Methods and Mixture Models
preprint

Greedy Empirical Bayes for Density Estimation

Edwin Fong, Taole Sha, Shibo Yu
preprint en

Abstract

Bayesian nonparametric density estimation is challenging in high dimensions, where traditional approaches like the Dirichlet process (DP) mixture rely on computationally intensive Markov chain Monte Carlo or approximations. We introduce the partition posterior bootstrap (PPB), where a flexible prior is constructed by smoothing a DP over a partition of the sample space. Conditioned on this partition, exact posterior sampling reduces to a simple Bayesian bootstrap-like algorithm. Importantly, we select the partition using a greedy empirical Bayes (GEB) algorithm which scales to high dimensions and large sample sizes. GEB greedily maximizes the closed-form marginal likelihood via local split decisions arising naturally from its factorization, yielding an adaptive binary tree partition. Our key theoretical result is that the GEB tree belongs to an admissible family with high probability, splitting all identifiable signals while avoiding over-splitting noise. As a result, the PPB posterior with the GEB tree attains adaptive contraction rates. Our theory holds under weak, verifiable assumptions, which we demonstrate in numerous examples. Empirically, PPB delivers accurate density estimates and uncertainty quantification at a fraction of the computational cost of other approaches, making it a practical off-the-shelf choice for high-dimensional Bayesian density estimation with theoretical guarantees.

arXiv (Cornell University)
Bayesian Methods and Mixture Models
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon