Efficient approximate Minimum-Rényi entropy couplings

The Rényi entropy is a natural generalization of Shannon entropy which depends on a parameter $ {{\alpha}} $. Given two probability distributions $ {\mathbf{p}} $ and $ {\mathbf{q}} $, the minimum Rényi entropy coupling is joint distribution of $ {\mathbf{p}} $ and $ {\mathbf{q}} $ with minimal Rényi entropy. We show in this paper that the greedy coupling proposed in Kocaoglu and Dimakis (2017) possesses a Rényi entropy exceeding the exact minimum value at most by $ g({{\alpha}}) $ bits. Here $ g({{\alpha}}) $ decreases to $ 0 $ as $ {{\alpha}} $ tends to infinite, $ g({{\alpha}})\leq 1 $ for $ {{\alpha}}\geq 0.3644 $ and $ g({{\alpha}}) $ tends to $ \log_2 (e)/e\approx 0.53 $ as $ {{\alpha}} $ tends to $ 1 $.

Authors

Institutions

Publication Details

Journal
Discrete and Continuous Dynamical Systems - S
Published
2025-01-01
DOI
https://doi.org/10.3934/dcdss.2025098
Primary Topic
Blind Source Separation Techniques
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Efficient approximate Minimum-Rényi entropy couplings

Feng Wang, Xian‐Yuan Wu, Yajing Ma
Discrete and Continuous Dynamical Systems - S
Blind Source Separation Techniques
article

Efficient approximate Minimum-Rényi entropy couplings

Feng Wang, Xian‐Yuan Wu, Yajing Ma
article en

Abstract

The Rényi entropy is a natural generalization of Shannon entropy which depends on a parameter $ {{\alpha}} $. Given two probability distributions $ {\mathbf{p}} $ and $ {\mathbf{q}} $, the minimum Rényi entropy coupling is joint distribution of $ {\mathbf{p}} $ and $ {\mathbf{q}} $ with minimal Rényi entropy. We show in this paper that the greedy coupling proposed in Kocaoglu and Dimakis (2017) possesses a Rényi entropy exceeding the exact minimum value at most by $ g({{\alpha}}) $ bits. Here $ g({{\alpha}}) $ decreases to $ 0 $ as $ {{\alpha}} $ tends to infinite, $ g({{\alpha}})\leq 1 $ for $ {{\alpha}}\geq 0.3644 $ and $ g({{\alpha}}) $ tends to $ \log_2 (e)/e\approx 0.53 $ as $ {{\alpha}} $ tends to $ 1 $.

Discrete and Continuous Dynamical Systems - SVol. 18(10)
Capital Normal University (CN)
National Natural Science Foundation of China
Openalex Percentile: Top 84%
Blind Source Separation Techniques
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.