## Short version

Gibbs's inequality is a simple inequality for real numbers, usually understood information-theoretically. In the jargon, it states that for two probability measures on a finite set, the relative entropy is always nonnegative.

I'd like to hear about non-information-theoretic ways of understanding it. I'd be particularly pleased if there were some nice geometric interpretation.

## Statement and proof of Gibbs's inequality

For natural numbers $n$, let $\mathbf{P}_n$ denote the set of probability
measures on an $n$-element set: that is,
$$
\mathbf{P}_n = \{ p \in \mathbb{R}^n : p_1, \ldots, p_n \geq 0, \sum p_i =
1 \}.
$$
**Theorem (Gibbs)** Let $p \in \mathbf{P}_n$. Then, for $q$
varying in $\mathbf{P}_n$, the quantity $\prod q_i^{p_i}$ is maximized by
$q = p$.

Usually this is stated in logarithmic form: $-\sum p_i \log q_i \geq -\sum p_i \log p_i$ for all $p, q \in \mathbf{P}_n$. But I'd like to reach a direct understanding of the product form.

There are at least two extremely easy proofs. Ignoring zero probabilities, they run as follows. The first: since $\log$ is concave, $\sum p_i \log (q_i/p_i) \leq \log \sum p_i (q_i/p_i) = 0$. The second: since $\log x \leq x - 1$ for all $x$, we have $\sum p_i \log (q_i/p_i) \leq \sum p_i (q_i/p_i - 1) = 0$.

## The question

Can Gibbs's inequality, in the product form stated above, be understood geometrically? Or if not geometrically, is there an intuitive interpretation other than the information-theoretic one? (I have nothing against information theory — it's just that I'd like to have multiple ways of thinking about it.)

There is a hint that Gibbs's inequality can be interpreted as some kind of isoperimetric inequality. Take $p$ to be the uniform distribution. Then the inequality states that for $q \in \mathbf{P}_n$, the quantity $(q_1 q_2 \cdots q_n)^{1/n}$ is maximized by taking $q$ to be uniform. We might as well remove the power $1/n$, and then the result is: among all $n$-dimensional boxes of prescribed total edge-length, the cube has the greatest volume.

But I see no way of extending the isoperimetric interpretation to non-uniform $p$. For example, take $p = (2/3, 1/3)$. Then Gibbs states that among all $q \in \mathbf{P}_2$, the maximum value of $q_1^2 q_2$ is attained by $q = (2/3, 1/3)$. This doesn't seem geometrically obvious to me in the way that the uniform case does.