Let us define the arithmetic, geometric, and harmonic means of $x,y \in \mathbb{R}$ weighted by $\alpha =(\alpha_x,\alpha_y) \in [0,1]$, respectively as

\begin{equation} a_\alpha(x,y) = \frac{\alpha_x x+\alpha_y y}{\alpha_x+\alpha_y} \end{equation}

\begin{equation} g_\alpha(x,y) = x^\frac{\alpha_x}{\alpha_x+\alpha_y}y^\frac{\alpha_y}{\alpha_x+\alpha_y} \end{equation}

\begin{equation} h_\alpha(x,y) = \frac{xy(\alpha_x+\alpha_y)}{\alpha_xy + \alpha_yx} \end{equation}

Let us define the following information quantities (see https://arxiv.org/pdf/1912.00610.pdf for more details on these quantities)

\begin{equation} A_\alpha(x,y) = \alpha_x \text{KL}\left(a_\alpha(x,y),x\right) + \alpha_y \text{KL}\left(a_\alpha(x,y),y\right) \end{equation}

\begin{equation} G_\alpha(x,y) = \alpha_x \text{KL}\left(g_\alpha(x,y),x\right) + \alpha_y \text{KL}\left(g_\alpha(x,y),y\right) \end{equation}

\begin{equation} H_\alpha(x,y) = \alpha_x \text{KL}\left(h_\alpha(x,y),x\right) + \alpha_y \text{KL}\left(h_\alpha(x,y),y\right) \end{equation} where $\text{KL}(\cdot)$ is the Kullback-Leibler divergence. The arithmetic-geometric-harmonic mean inequality is a very well known result that states

\begin{equation} a_\alpha(x,y) \geq g_\alpha(x,y) \geq h_\alpha(x,y) \end{equation} $\forall x,y \in \mathbb{R}$.

Given this inequality, and given that $x\geq y$, can we conclude that the the following is true?

\begin{equation} A_\alpha(x,y) \geq G_\alpha(x,y) \geq H_\alpha(x,y) \end{equation}

  • $\begingroup$ In the case of simple distributions such as Bernoulli, Poisson or exponential distributions, for which the KL admits a close form, I think it should follow directly from the monotonicity of the function $\log(\cdot)$. But is this true for any distribution? How to prove it? $\endgroup$
    – Apprentice
    Apr 17, 2020 at 9:26


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.