1
$\begingroup$

Gibbs' inequality is equivalent to:

\begin{equation} \sum_{i} \ln q_i^{p_i}-\ln p_i^{p_i} \leq 0 \end{equation}

where $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$.

Now, a friend of mine suggested that assuming $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$, Gibbs' inequality implies:

\begin{equation} \sum_{i} q_i^{p_i}-p_i^{p_i} \leq 0 \end{equation}

Right now I doubt this is true but I can't think of a counter-example.

Update: I have found experimental evidence for my friends' conjecture by running the following Python code:

import numpy as np

count = 0

for i in range(10000):
    # randomly create distributions:
    P, Q = np.random.rand(10), np.random.rand(10)
    p, q = P/np.sum(P), Q/np.sum(Q)

    M = np.sum([p[i]**p[i] for i in range(10)])
    m = np.sum([q[i]**p[i] for i in range(10)])

    if m <= M:
        count+=1

The inequality was satisfied every single time I ran this script.

$\endgroup$
  • 1
    $\begingroup$ your $\geq$ should be $\leq$. $\endgroup$ – Carlo Beenakker Sep 28 '17 at 11:49
  • $\begingroup$ @CarloBeenakker Thank you for the clarification. $\endgroup$ – Aidan Rocke Sep 28 '17 at 12:08
  • 1
    $\begingroup$ If, say, $p_1^{p_1} > p_2^{p_2}$ then taking $q_1 = p_1 + \varepsilon$ and $q_2 = p_2 - \varepsilon$ (and $q_i = p_i$ for $i>2$) yields a counter-example to this inequality for $\varepsilon$ small enough... $\endgroup$ – js21 Sep 28 '17 at 12:27
5
$\begingroup$

That does not seem to be true. Here is how you can build a counterexample: Define $F(q) = \sum_i q_i^{p_i} - p_i^{p_i}$ and note that $F(p)=0$. To find $q$ such that $F(q)> 0$ try to set $\tilde q = p + t \nabla F(p) = p+tp^p$ (exponentiation applied componentwise) for some small $t$, and renormalize to get $q = \frac{\tilde q}{\sum_i \tilde q_i}$. I used a random $p$, $t=0.01$ and got

p =
 0.1532696579656778
 0.2067195573676092
 0.0459491849155964
 0.1571139939276139
 0.0425424874234802
 0.0322607550224537
 0.2054340486142850
 0.0106719016346504
 0.1310342405827529
 0.0150041725458808

and

q =

 0.1485356620020452
 0.1976565884683083
 0.0504718308948335
 0.1520644631305625
 0.0473824833517361
 0.0380756305476259
 0.1964738847480934
 0.0186616781313398 
 0.1281407424548285
 0.0225370362706270

with $F(q) \approx 0.001>0$

$\endgroup$
  • 2
    $\begingroup$ Nice general sanity check for inequalities! The gradient must vanish where equality is achieved. $\endgroup$ – Tim Carson Oct 2 '17 at 15:29
2
$\begingroup$

Denote $x_i=p_i^{p_i}, y_i=q_i^{p_i}$. Then your claim is that whenever $\sum \ln x_i-\ln y_i\leqslant 0$, we have $\sum x_i-y_i\leqslant 0$ (well, $x_i$ and $y_i$ are not arbitrary numbers, but they may take arbitrary values on the interval $[e^{-1/e},1)$: first find $p_i\in (0,1)$ knowing $x_i$, then find $q_i$ knowing $y_i$ and $p_i$.) Of course this is not true. Indeed, imagine that $\sum x_i=\sum y_i$, but $\sum \ln x_i<\sum \ln y_i$. Increase $x_1$ a bit.

$\endgroup$
  • $\begingroup$ There may be a slight misunderstanding on the question. The original question was of the form "does $A$ imply $B$ ?". Since $A$ is known to hold here this is equivalent to "does $B$ hold ?". $\endgroup$ – js21 Sep 28 '17 at 13:12
  • $\begingroup$ I understood it so that we do not require $\sum p_i=\sum q_i=1$. But I may be wrong. $\endgroup$ – Fedor Petrov Sep 28 '17 at 17:08
  • 1
    $\begingroup$ @FedorPetrov That assumption is actually fundamental to Gibbs' Inequality. It's also explicitly mentioned in the question details. $\endgroup$ – Aidan Rocke Oct 1 '17 at 20:16
  • $\begingroup$ yes, I see now. But then the formulation looks bit strange, as js21 remarks. $\endgroup$ – Fedor Petrov Oct 1 '17 at 20:45
  • 1
    $\begingroup$ yes, now the formulation looks ok $\endgroup$ – Fedor Petrov Oct 2 '17 at 14:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.