Skip to main content
3 of 5
added 328 characters in body
john mangual
  • 22.8k
  • 4
  • 63
  • 172

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$


This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.

john mangual
  • 22.8k
  • 4
  • 63
  • 172