2
$\begingroup$

Let $X^1 \ldots X^n$ be random variables on $\mathbb{R}^d$ with an arbitrary joint probability distribution $\mu$ on $\mathbb{R}^{n \times d}$. Let $\nu = \nu^1 \times \ldots \times \nu^n$ be a product of $n$ distributions, where each $\nu^i$ is a distribution on $\mathbb{R}^d$. Finally let $\mu^i$ denote the marginal distribution of $X^i$.

The following is well-known to hold by Csiszar [1]:

$$\sum_{i=1}^n \mathsf{KL}(\mu^i \lVert \nu^i) \leq \mathsf{KL}(\mu \lVert \nu) $$

Does a variant of this inequality hold for the Renyi divergence? That is, define $\mathsf{R}_\alpha(\mu \Vert \nu) = \frac{1}{\alpha-1} \log \int \left(\frac{\text{d} \mu}{\text{d} \nu}\right)^{\alpha}\, \nu$, then is it known whether $$ \sum_{i=1}^n \mathsf{R}_\alpha(\mu^i \Vert \nu^i) \leq c \mathsf{R}_\alpha(\mu \Vert \nu)$$ for some constant $c < \infty$ that does not depend on $n$ (it may depend on $\alpha$)?

For simplicity, we can also let $\mu \ll \nu$, $\mu^i \ll \nu^i$ as well.

$\endgroup$
1
  • $\begingroup$ A weaker inequality $ \sum_{i=1}^n \mathsf{R}_\alpha(\mu^i \Vert \nu^i) \leq c \mathsf{R}_{c'\alpha}(\mu \Vert \nu)$ would also be helpful, for some $c'$ not too large. $\endgroup$ Commented Feb 15 at 20:37

0

You must log in to answer this question.

Browse other questions tagged .