# Inequality on the Kullback-Leibler divergence

Let us define the arithmetic, geometric, and harmonic means of $$x,y \in \mathbb{R}$$ weighted by $$\alpha =(\alpha_x,\alpha_y) \in [0,1]$$, respectively as

$$$$a_\alpha(x,y) = \frac{\alpha_x x+\alpha_y y}{\alpha_x+\alpha_y}$$$$

$$$$g_\alpha(x,y) = x^\frac{\alpha_x}{\alpha_x+\alpha_y}y^\frac{\alpha_y}{\alpha_x+\alpha_y}$$$$

$$$$h_\alpha(x,y) = \frac{xy(\alpha_x+\alpha_y)}{\alpha_xy + \alpha_yx}$$$$

Let us define the following information quantities (see https://arxiv.org/pdf/1912.00610.pdf for more details on these quantities)

$$$$A_\alpha(x,y) = \alpha_x \text{KL}\left(a_\alpha(x,y),x\right) + \alpha_y \text{KL}\left(a_\alpha(x,y),y\right)$$$$

$$$$G_\alpha(x,y) = \alpha_x \text{KL}\left(g_\alpha(x,y),x\right) + \alpha_y \text{KL}\left(g_\alpha(x,y),y\right)$$$$

$$$$H_\alpha(x,y) = \alpha_x \text{KL}\left(h_\alpha(x,y),x\right) + \alpha_y \text{KL}\left(h_\alpha(x,y),y\right)$$$$ where $$\text{KL}(\cdot)$$ is the Kullback-Leibler divergence. The arithmetic-geometric-harmonic mean inequality is a very well known result that states

$$$$a_\alpha(x,y) \geq g_\alpha(x,y) \geq h_\alpha(x,y)$$$$ $$\forall x,y \in \mathbb{R}$$.

Given this inequality, and given that $$x\geq y$$, can we conclude that the the following is true?

$$$$A_\alpha(x,y) \geq G_\alpha(x,y) \geq H_\alpha(x,y)$$$$

• In the case of simple distributions such as Bernoulli, Poisson or exponential distributions, for which the KL admits a close form, I think it should follow directly from the monotonicity of the function $\log(\cdot)$. But is this true for any distribution? How to prove it? Apr 17, 2020 at 9:26