1
$\begingroup$

Gibbs' inequality is equivalent to:

\begin{equation} \sum_{i} \ln q_i^{p_i}-\ln p_i^{p_i} \leq 0 \end{equation}

where $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$.

Now, a friend of mine suggested that assuming $p_i,q_i \in [0,1]$ and $\sum_i p_i = \sum_i q_i=1$, Gibbs' inequality implies:

\begin{equation} \sum_{i} q_i^{p_i}-p_i^{p_i} \leq 0 \end{equation}

Right now I doubt this is true but I can't think of a counter-example.

Update: I have found experimental evidence for my friends' conjecture by running the following Python code:

import numpy as np

count = 0

for i in range(10000):
    # randomly create distributions:
    P, Q = np.random.rand(10), np.random.rand(10)
    p, q = P/np.sum(P), Q/np.sum(Q)

    M = np.sum([p[i]**p[i] for i in range(10)])
    m = np.sum([q[i]**p[i] for i in range(10)])

    if m <= M:
        count+=1

The inequality was satisfied every single time I ran this script.

$\endgroup$
3
  • 1
    $\begingroup$ your $\geq$ should be $\leq$. $\endgroup$ Commented Sep 28, 2017 at 11:49
  • $\begingroup$ @CarloBeenakker Thank you for the clarification. $\endgroup$ Commented Sep 28, 2017 at 12:08
  • 1
    $\begingroup$ If, say, $p_1^{p_1} > p_2^{p_2}$ then taking $q_1 = p_1 + \varepsilon$ and $q_2 = p_2 - \varepsilon$ (and $q_i = p_i$ for $i>2$) yields a counter-example to this inequality for $\varepsilon$ small enough... $\endgroup$
    – js21
    Commented Sep 28, 2017 at 12:27

2 Answers 2

5
$\begingroup$

That does not seem to be true. Here is how you can build a counterexample: Define $F(q) = \sum_i q_i^{p_i} - p_i^{p_i}$ and note that $F(p)=0$. To find $q$ such that $F(q)> 0$ try to set $\tilde q = p + t \nabla F(p) = p+tp^p$ (exponentiation applied componentwise) for some small $t$, and renormalize to get $q = \frac{\tilde q}{\sum_i \tilde q_i}$. I used a random $p$, $t=0.01$ and got

p =
 0.1532696579656778
 0.2067195573676092
 0.0459491849155964
 0.1571139939276139
 0.0425424874234802
 0.0322607550224537
 0.2054340486142850
 0.0106719016346504
 0.1310342405827529
 0.0150041725458808

and

q =

 0.1485356620020452
 0.1976565884683083
 0.0504718308948335
 0.1520644631305625
 0.0473824833517361
 0.0380756305476259
 0.1964738847480934
 0.0186616781313398 
 0.1281407424548285
 0.0225370362706270

with $F(q) \approx 0.001>0$

$\endgroup$
1
  • 2
    $\begingroup$ Nice general sanity check for inequalities! The gradient must vanish where equality is achieved. $\endgroup$
    – Tim Carson
    Commented Oct 2, 2017 at 15:29
2
$\begingroup$

Denote $x_i=p_i^{p_i}, y_i=q_i^{p_i}$. Then your claim is that whenever $\sum \ln x_i-\ln y_i\leqslant 0$, we have $\sum x_i-y_i\leqslant 0$ (well, $x_i$ and $y_i$ are not arbitrary numbers, but they may take arbitrary values on the interval $[e^{-1/e},1)$: first find $p_i\in (0,1)$ knowing $x_i$, then find $q_i$ knowing $y_i$ and $p_i$.) Of course this is not true. Indeed, imagine that $\sum x_i=\sum y_i$, but $\sum \ln x_i<\sum \ln y_i$. Increase $x_1$ a bit.

$\endgroup$
6
  • $\begingroup$ There may be a slight misunderstanding on the question. The original question was of the form "does $A$ imply $B$ ?". Since $A$ is known to hold here this is equivalent to "does $B$ hold ?". $\endgroup$
    – js21
    Commented Sep 28, 2017 at 13:12
  • $\begingroup$ I understood it so that we do not require $\sum p_i=\sum q_i=1$. But I may be wrong. $\endgroup$ Commented Sep 28, 2017 at 17:08
  • 1
    $\begingroup$ @FedorPetrov That assumption is actually fundamental to Gibbs' Inequality. It's also explicitly mentioned in the question details. $\endgroup$ Commented Oct 1, 2017 at 20:16
  • $\begingroup$ yes, I see now. But then the formulation looks bit strange, as js21 remarks. $\endgroup$ Commented Oct 1, 2017 at 20:45
  • 1
    $\begingroup$ yes, now the formulation looks ok $\endgroup$ Commented Oct 2, 2017 at 14:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .