On the entropy growth of sums of iid discrete random variables
We derive an asymptotic lower bound on the Shannon entropy $H$ of sums of $N$ arbitrary iid discrete random variables. The derived bound $H \geq \frac{r(X)}{2}\log(N) + {\it cst}$ is given in terms of the incommensurability rank $r(X)$ of the random variable -- a positive integer quantity that we introduce. The derivation does not rely on central limit theorems, but builds upon the known expressions of the asymptotic entropy of the multinomial distribution and sums of iid lattice random variables, which correspond to the case $r(X)=1$.
Publication Details
- Journal
- ArXiv.org
- Published
- 2025-08-07
- Primary Topic
- Wireless Communication Security Techniques
- Type
- article
- Field-Weighted Citation Impact
- 0.00