# A corollary of Gibbs' inequality

Gibbs' inequality is equivalent to:

$$\sum_{i} \ln q_i^{p_i}-\ln p_i^{p_i} \leq 0$$

where $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$.

Now, a friend of mine suggested that assuming $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$, Gibbs' inequality implies:

$$\sum_{i} q_i^{p_i}-p_i^{p_i} \leq 0$$

Right now I doubt this is true but I can't think of a counter-example.

Update: I have found experimental evidence for my friends' conjecture by running the following Python code:

import numpy as np

count = 0

for i in range(10000):
# randomly create distributions:
P, Q = np.random.rand(10), np.random.rand(10)
p, q = P/np.sum(P), Q/np.sum(Q)

M = np.sum([p[i]**p[i] for i in range(10)])
m = np.sum([q[i]**p[i] for i in range(10)])

if m <= M:
count+=1


The inequality was satisfied every single time I ran this script.

• your $\geq$ should be $\leq$. – Carlo Beenakker Sep 28 '17 at 11:49
• @CarloBeenakker Thank you for the clarification. – Aidan Rocke Sep 28 '17 at 12:08
• If, say, $p_1^{p_1} > p_2^{p_2}$ then taking $q_1 = p_1 + \varepsilon$ and $q_2 = p_2 - \varepsilon$ (and $q_i = p_i$ for $i>2$) yields a counter-example to this inequality for $\varepsilon$ small enough... – js21 Sep 28 '17 at 12:27

That does not seem to be true. Here is how you can build a counterexample: Define $F(q) = \sum_i q_i^{p_i} - p_i^{p_i}$ and note that $F(p)=0$. To find $q$ such that $F(q)> 0$ try to set $\tilde q = p + t \nabla F(p) = p+tp^p$ (exponentiation applied componentwise) for some small $t$, and renormalize to get $q = \frac{\tilde q}{\sum_i \tilde q_i}$. I used a random $p$, $t=0.01$ and got

p =
0.1532696579656778
0.2067195573676092
0.0459491849155964
0.1571139939276139
0.0425424874234802
0.0322607550224537
0.2054340486142850
0.0106719016346504
0.1310342405827529
0.0150041725458808


and

q =

0.1485356620020452
0.1976565884683083
0.0504718308948335
0.1520644631305625
0.0473824833517361
0.0380756305476259
0.1964738847480934
0.0186616781313398
0.1281407424548285
0.0225370362706270


with $F(q) \approx 0.001>0$

• Nice general sanity check for inequalities! The gradient must vanish where equality is achieved. – Tim Carson Oct 2 '17 at 15:29

Denote $x_i=p_i^{p_i}, y_i=q_i^{p_i}$. Then your claim is that whenever $\sum \ln x_i-\ln y_i\leqslant 0$, we have $\sum x_i-y_i\leqslant 0$ (well, $x_i$ and $y_i$ are not arbitrary numbers, but they may take arbitrary values on the interval $[e^{-1/e},1)$: first find $p_i\in (0,1)$ knowing $x_i$, then find $q_i$ knowing $y_i$ and $p_i$.) Of course this is not true. Indeed, imagine that $\sum x_i=\sum y_i$, but $\sum \ln x_i<\sum \ln y_i$. Increase $x_1$ a bit.

• There may be a slight misunderstanding on the question. The original question was of the form "does $A$ imply $B$ ?". Since $A$ is known to hold here this is equivalent to "does $B$ hold ?". – js21 Sep 28 '17 at 13:12
• I understood it so that we do not require $\sum p_i=\sum q_i=1$. But I may be wrong. – Fedor Petrov Sep 28 '17 at 17:08
• @FedorPetrov That assumption is actually fundamental to Gibbs' Inequality. It's also explicitly mentioned in the question details. – Aidan Rocke Oct 1 '17 at 20:16
• yes, I see now. But then the formulation looks bit strange, as js21 remarks. – Fedor Petrov Oct 1 '17 at 20:45
• yes, now the formulation looks ok – Fedor Petrov Oct 2 '17 at 14:49