4
$\begingroup$

If $p(x)$ is a discrete probabilistic density function, one could construct another discrete probabilistic density function proportional to $p(x)[1-p(x)]$ with a corresponding partition function to make the new function sum to $1$. (Rule out the degenerate case where $p(x_0) = 1$ for a unique $x_0$.)

It seems that the new constructed p.d.f has a bigger or equal entropy than $p(x)$. How to prove that? Suggestions are welcomed for any references or direct solution.

$\endgroup$
3
  • 4
    $\begingroup$ There is something wrong in the setup: a probability density function may take values more than $1$. And without some continuity assumption on $p$ we could have $p(x) \in \{0,1\}$ for all $x$, so $p(x)(1-p(x)) = 0$ always. $\endgroup$ Commented Jan 12, 2019 at 8:58
  • 1
    $\begingroup$ I guess that density is a misleading word here: $p(x) $ is a measure of atom $\{x\} $. $\endgroup$ Commented Jan 12, 2019 at 9:35
  • 1
    $\begingroup$ Ah: I see. I somehow misread 'finite support' as 'compact support'. I'm going to edit the question to make it clear that the density is discrete. $\endgroup$ Commented Jan 12, 2019 at 9:40

1 Answer 1

8
$\begingroup$

Denote $f(p)=p(1-p)$, $H(p)=-p\log p$, let $p_1,\dots,p_n$ denote all positive probabilities of our distribution, then $\sum p_i=1$, finally denote $s=\sum_i f(p_i)$. Then we need to prove the inequality $$ \sum_i H(f(p_i)/s)\geqslant \sum_i H(p_i). $$ Since $H$ is concave, it suffices to prove that the multiset $\{p_1,\dots,p_n\}$ majorizes the multiset $\{f(p_1)/s,f(p_2)/s,\dots,f(p_n)/s\}$. We may assume that $p_1\geqslant p_2\geqslant \ldots\geqslant p_n$, it implies $f(p_1)\geqslant f(p_2)\geqslant \ldots\geqslant f(p_n)$, since $f(p_i)-f(p_j)=(p_i-p_j)(1-p_i-p_j)$, and $1-p_i-p_j\geqslant 0$ for $i\ne j$. Therefore we need to check for every $k$ that $$\frac{p_1+\dots+p_k}{p_1+\dots+p_n}=p_1+\dots+p_k\geqslant \frac{f(p_1)+\dots+f(p_k)}s,$$ that is equivalent (by multiplying through by the denominators on the far left and far right) to $$(p_1+\dots+p_k)(p_{k+1}^2+\dots+p_n^2)\leqslant (p_1^2+\dots+p_k^2)(p_{k+1}+\dots+p_n),$$ which is a sum of obvious inequalities $p_ip_j^2\leqslant p_i^2 p_j$ over all pairs $i\leqslant k<j$.

$\endgroup$
2
  • $\begingroup$ thanks a lot. Is there a theory about such probability transform? $\endgroup$
    – sunxd
    Commented Jan 16, 2019 at 22:43
  • $\begingroup$ @sunxd no idea about such theory $\endgroup$ Commented Jan 17, 2019 at 14:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .