Efficient approximate Minimum-Rényi entropy couplings
The Rényi entropy is a natural generalization of Shannon entropy which depends on a parameter $ {{\alpha}} $. Given two probability distributions $ {\mathbf{p}} $ and $ {\mathbf{q}} $, the minimum Rényi entropy coupling is joint distribution of $ {\mathbf{p}} $ and $ {\mathbf{q}} $ with minimal Rényi entropy. We show in this paper that the greedy coupling proposed in Kocaoglu and Dimakis (2017) possesses a Rényi entropy exceeding the exact minimum value at most by $ g({{\alpha}}) $ bits. Here $ g({{\alpha}}) $ decreases to $ 0 $ as $ {{\alpha}} $ tends to infinite, $ g({{\alpha}})\leq 1 $ for $ {{\alpha}}\geq 0.3644 $ and $ g({{\alpha}}) $ tends to $ \log_2 (e)/e\approx 0.53 $ as $ {{\alpha}} $ tends to $ 1 $.
Authors
- Feng Wang (ORCID: https://orcid.org/0000-0002-4591-3888)
- Xian‐Yuan Wu
- Yajing Ma
Institutions
- Capital Normal University (CN)
Publication Details
- Journal
- Discrete and Continuous Dynamical Systems - S
- Published
- 2025-01-01
- DOI
- https://doi.org/10.3934/dcdss.2025098
- Primary Topic
- Blind Source Separation Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Natural Science Foundation of China