Let $X,Y$ be probability measures on $\{1,2,\dots,n\}$. For $0\le \alpha \le 1$, set $K=\sum_i X(i)^\alpha Y(i)^{1-\alpha}$ so that $Z:=\frac{1}{K}X^\alpha Y^{1-\alpha}$ is also a probability measure on $\{1,2,\dots,n\}$. How can we prove the inequality $$\alpha H(X)+ (1-\alpha) H(Y)\geq K^2 H(Z),$$ where $H(X)=-\sum_{i=1}^n X(i)\log X(i)$ is the entropy function?

This is a small generalization of a recent question. The generalization is supported by a modest amount of numerical experimentation.

[I hope it's ok to post this generalization as a separate question. I'm not allowed to comment on the previous question, and this isn't an answer, so I didn't see an alternative.]

I've tried to translate the inequality into physics language (classical statistical mechanics), but I don't see a physical meaning.

Suppose $H$ is a hamiltonian --- a self-adjoint operator acting on a Hilbert space, of finite dimension $n$, for simplicity. Write its eigenvalues $E_i$, $i=1,\ldots,n$. The partition function is $$Z(H) = \mathrm{tr}\; e^{-H} = \sum_i e^{-E_i}.$$ The density matrix is $$\rho = Z^{-1} e^{-H}.$$ (In the language of the original question, the probability measure is $X(i) = Z^{-1} e^{-E_i}$.)

The entropy is $$S(H) = - \mathrm{tr}\, (\rho \ln \rho).$$

Let $H_0$ and $H_1$ be two commuting hamiltonians. so they can be diagonalized simultaneously. Let $$H_\alpha = (1-\alpha) H_0 + \alpha H_1\,.$$ The conjectured inequality is $$(1-\alpha)S(H_0) + \alpha S(H_1) \ge K^2 S(H_\alpha)$$ where $$K = \frac{Z(H_\alpha)}{Z(H_0)^{1-\alpha}Z(H_1)^{\alpha}}.$$ I don't see a physical interpretation for the factor $K$.

Does the inequality hold without the assumption that $H_0$ and $H_1$ commute, i.e. in quantum statistical mechanics?

  • $\begingroup$ Does it hold even with the commuting assumption? Could you please write everything using matrices also in the main question (it makes it much easier to parse for people like me); thanks! $\endgroup$ – Suvrit Aug 2 '13 at 16:14
  • $\begingroup$ The main question is written in the language of the earlier question "An Entropy Inequality" (linked above). When $H_0$ and $H_1$ are commuting matrices, diagonalize them simultaneously. $H_0$ has eigenvalues $E_{0i}$ and $H_1$ has eigenvalues $E_{1i}$. The probability measures are $X(i)=e^{-E_{1i}}/Z(H_1)$ and $Y(i)=e^{-E_{0i}}/Z(H_0)$. $H_\alpha$ has eigenvalues $E_{\alpha i}=(1-\alpha) E_{0i}+\alpha E_{1i}$. $Z(i)= e^{-E_{\alpha i}}/Z(H_\alpha)$. Substituting, you'll see that the two versions of the inequality are the same. $\endgroup$ – Daniel Friedan Aug 2 '13 at 17:01

Not an answer, but a random thought This looks very close to the Lieb/Wigner/Yamase inequality, as in these very nice notes of Eric Carlen's.

  • 1
    $\begingroup$ Could you elaborate? I don't see the connection (though I'm far from expert). $\endgroup$ – Daniel Friedan Aug 2 '13 at 16:33

Try: $-H(Z) = \mathbb{E} [\log Z] $ we have an equality:

$$ H(Z) = -\mathbb{E} [\log \tfrac{1}{K}] - \alpha \mathbb{E} [\log X]-(1-\alpha) \mathbb{E} [\log Y] = \log K + \alpha H(X) +(1-\alpha) H(Y)$$

rearrange a bit to look like the original problem:

$$ \alpha H(X) +(1-\alpha) H(Y) = H(Z) - \log K $$

$K \leq 1$ using Hölder inequality and so $H(Z) \geq 0$ and $\log K < 0$ is negative, as in a related question on entropy inequalities

$$ H(Z) - \log K \geq H(Z) \geq K^2H(Z)$$

This is not correct as stated. In fact, $-H(Z) = \mathbb{E}_Z [\log Z]$ with respect to the probability measure $Z$.

Entropy $H$ is a concave functional of the measures:

$$ H( t X + (1-t)Y) \geq H(t X + (1-t)Y|T) = tH(X) + (1-t)H(Y) $$

here $T$ is a 0-1 Bernoulli random variable with $\mathbb{P}(T=1) = t$.

I am really going to drop it after this. The way to prove one entropy is lower than another is to use conditional entropy: $H(X) \geq H(X|Y)$

What to condition on? I use unorthodox and possibly wrong notation:

$$ tH(X) + (1-t)H(Y) = \mathbb{E}[t \log X + (1-t)\log Y | X,Y,t]$$

Given distributions $X,Y$ and Bernoulli random variable $t$ we can construct the above entropy.

$$ K^2H(Z) = \mathbb{E}[\sqrt{XY}\,\big|i]^2 \mathbb{E}[t \log X + (1-t)\log Y \,\big| Z=\tfrac{\sqrt{XY}}{K},t]$$

Where $i$ is uniform in $[1,2,\dots, n]$. It still doesn't look right.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.