Return to Revisions

For the requested examples see Rényi Divergence and Kullback-Leibler Divergence (2012).

• Two continuous distributions: Equation 10 gives the Rényi divergence between two Gaussian distributions (mean $\mu_i$, variance $\sigma_i$):

$$ D_\alpha\Big({\cal N}(\mu_0,\sigma_0^2)\|{\cal N}(\mu_1,\sigma_1^2)\Big) = \frac{\alpha(\mu_1 - \mu_0)^2}{2\sigma_\alpha^2} + \frac{1}{1-\alpha} \log\left( \frac{\sigma_\alpha}{\sigma_0^{1-\alpha}\sigma_1^\alpha}\right), $$ for $\sigma_\alpha^2 = (1-\alpha)\sigma_0^2 + \alpha \sigma_1^2 > 0$

• Two discrete distributions: Figures 2 and 3 show the Rényi divergence $D_\alpha(P||Q)$ for fixed $Q$ as $P$ varies over a sample space containing two or three elements.