Bounding Excess Minimum Risk via Rényi's Divergence

Given finite dimensional random vectors <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Y,X}$</tex> and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Z}$</tex> that form a Markov chain in that order <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$(\boldsymbol{Y}\rightarrow \boldsymbol{X}\rightarrow \boldsymbol{Z})$</tex>, we derive Rényi divergence based upper bounds for excess minimum risk, where <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Y}$</tex> is a (target) vector that is to be estimated from an observed (feature) vector <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{X}$</tex> or its (stochastically) degraded version <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Z}$</tex>. We define the excess minimum risk as the difference between the minimum expected loss in estimating <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Y}$</tex> from <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{X}$</tex> and the minimum expected loss in estimating <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Y}$</tex> from <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Z}$</tex>. We obtain a family of bounds which generalize the bounds developed by Györfi et al. (2023) expressed in terms of Shannon's mutual information. Our bounds are similar to the bounds by Modak et al. (2021) obtained in the context of the generalization error of learning algorithms, but unlike the latter they do not involve fixed sub-Gaussian parameters and therefore hold for more general joint distributions of <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Y,X}$</tex>, and <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$\boldsymbol{Z}$</tex>. We also provide an example with Bernoulli random variables where Rényi's divergence based upper bound are tighter than mutual information bounds.