1
$\begingroup$

Gibbs' inequality is equivalent to:

\begin{equation} \sum_{i} \ln q_i^{p_i}-\ln p_i^{p_i} \leq 0 \end{equation}

where $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$.

Now, a friend of mine suggested that assuming $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$, Gibbs' inequality implies:

\begin{equation} \sum_{i} q_i^{p_i}-p_i^{p_i} \leq 0 \end{equation}

Right now I doubt this is true but I can't think of a counter-example.

Update: I have found experimental evidence for my friends' conjecture by running the following Python code:

import numpy as np

count = 0

for i in range(10000):
    # randomly create distributions:
    P, Q = np.random.rand(10), np.random.rand(10)
    p, q = P/np.sum(P), Q/np.sum(Q)

    M = np.sum([p[i]**p[i] for i in range(10)])
    m = np.sum([q[i]**p[i] for i in range(10)])

    if m <= M:
        count+=1

The inequality was satisfied every single time I ran this script.

$\endgroup$
3
  • 1
    $\begingroup$ your $\geq$ should be $\leq$. $\endgroup$ Sep 28, 2017 at 11:49
  • $\begingroup$ @CarloBeenakker Thank you for the clarification. $\endgroup$ Sep 28, 2017 at 12:08
  • 1
    $\begingroup$ If, say, $p_1^{p_1} > p_2^{p_2}$ then taking $q_1 = p_1 + \varepsilon$ and $q_2 = p_2 - \varepsilon$ (and $q_i = p_i$ for $i>2$) yields a counter-example to this inequality for $\varepsilon$ small enough... $\endgroup$
    – js21
    Sep 28, 2017 at 12:27

2 Answers 2

5
$\begingroup$

That does not seem to be true. Here is how you can build a counterexample: Define $F(q) = \sum_i q_i^{p_i} - p_i^{p_i}$ and note that $F(p)=0$. To find $q$ such that $F(q)> 0$ try to set $\tilde q = p + t \nabla F(p) = p+tp^p$ (exponentiation applied componentwise) for some small $t$, and renormalize to get $q = \frac{\tilde q}{\sum_i \tilde q_i}$. I used a random $p$, $t=0.01$ and got

p =
 0.1532696579656778
 0.2067195573676092
 0.0459491849155964
 0.1571139939276139
 0.0425424874234802
 0.0322607550224537
 0.2054340486142850
 0.0106719016346504
 0.1310342405827529
 0.0150041725458808

and

q =

 0.1485356620020452
 0.1976565884683083
 0.0504718308948335
 0.1520644631305625
 0.0473824833517361
 0.0380756305476259
 0.1964738847480934
 0.0186616781313398 
 0.1281407424548285
 0.0225370362706270

with $F(q) \approx 0.001>0$

$\endgroup$
1
  • 2
    $\begingroup$ Nice general sanity check for inequalities! The gradient must vanish where equality is achieved. $\endgroup$
    – Tim Carson
    Oct 2, 2017 at 15:29
2
$\begingroup$

Denote $x_i=p_i^{p_i}, y_i=q_i^{p_i}$. Then your claim is that whenever $\sum \ln x_i-\ln y_i\leqslant 0$, we have $\sum x_i-y_i\leqslant 0$ (well, $x_i$ and $y_i$ are not arbitrary numbers, but they may take arbitrary values on the interval $[e^{-1/e},1)$: first find $p_i\in (0,1)$ knowing $x_i$, then find $q_i$ knowing $y_i$ and $p_i$.) Of course this is not true. Indeed, imagine that $\sum x_i=\sum y_i$, but $\sum \ln x_i<\sum \ln y_i$. Increase $x_1$ a bit.

$\endgroup$
6
  • $\begingroup$ There may be a slight misunderstanding on the question. The original question was of the form "does $A$ imply $B$ ?". Since $A$ is known to hold here this is equivalent to "does $B$ hold ?". $\endgroup$
    – js21
    Sep 28, 2017 at 13:12
  • $\begingroup$ I understood it so that we do not require $\sum p_i=\sum q_i=1$. But I may be wrong. $\endgroup$ Sep 28, 2017 at 17:08
  • 1
    $\begingroup$ @FedorPetrov That assumption is actually fundamental to Gibbs' Inequality. It's also explicitly mentioned in the question details. $\endgroup$ Oct 1, 2017 at 20:16
  • $\begingroup$ yes, I see now. But then the formulation looks bit strange, as js21 remarks. $\endgroup$ Oct 1, 2017 at 20:45
  • 1
    $\begingroup$ yes, now the formulation looks ok $\endgroup$ Oct 2, 2017 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.