An Entropy Inequality (generalized)

Question

Let $X,Y$ be probability measures on $\{1,2,\dots,n\}$. For $0\le \alpha \le 1$, set $K=\sum_i X(i)^\alpha Y(i)^{1-\alpha}$ so that $Z:=\frac{1}{K}X^\alpha Y^{1-\alpha}$ is also a probability measure on $\{1,2,\dots,n\}$. How can we prove the inequality $$\alpha H(X)+ (1-\alpha) H(Y)\geq K^2 H(Z),$$ where $H(X)=-\sum_{i=1}^n X(i)\log X(i)$ is the entropy function?

This is a small generalization of a recent question. The generalization is supported by a modest amount of numerical experimentation.

[I hope it's ok to post this generalization as a separate question. I'm not allowed to comment on the previous question, and this isn't an answer, so I didn't see an alternative.]

I've tried to translate the inequality into physics language (classical statistical mechanics), but I don't see a physical meaning.

Suppose $H$ is a hamiltonian --- a self-adjoint operator acting on a Hilbert space, of finite dimension $n$, for simplicity. Write its eigenvalues $E_i$, $i=1,\ldots,n$. The partition function is $$Z(H) = \mathrm{tr}\; e^{-H} = \sum_i e^{-E_i}.$$ The density matrix is $$\rho = Z^{-1} e^{-H}.$$ (In the language of the original question, the probability measure is $X(i) = Z^{-1} e^{-E_i}$.)

The entropy is $$S(H) = - \mathrm{tr}\, (\rho \ln \rho).$$

Let $H_0$ and $H_1$ be two commuting hamiltonians. so they can be diagonalized simultaneously. Let $$H_\alpha = (1-\alpha) H_0 + \alpha H_1\,.$$ The conjectured inequality is $$(1-\alpha)S(H_0) + \alpha S(H_1) \ge K^2 S(H_\alpha)$$ where $$K = \frac{Z(H_\alpha)}{Z(H_0)^{1-\alpha}Z(H_1)^{\alpha}}.$$ I don't see a physical interpretation for the factor $K$.

Does the inequality hold without the assumption that $H_0$ and $H_1$ commute, i.e. in quantum statistical mechanics?

Answer 1

Not an answer, but a random thought This looks very close to the Lieb/Wigner/Yamase inequality, as in these very nice notes of Eric Carlen's.

Answer 2

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$

---

This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.

---

I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.