5
$\begingroup$

If $p(x)$ is a discrete probabilistic density function, one could construct another discrete probabilistic density function proportional to $p(x)[1-p(x)]$ with a corresponding partition function to make the new function sum to $1$. (Rule out the degenerate case where $p(x_0) = 1$ for a unique $x_0$.)

It seems that the new constructed p.d.f has a bigger or equal entropy than $p(x)$. How to prove that? Suggestions are welcomed for any references or direct solution.

$\endgroup$
  • 4
    $\begingroup$ There is something wrong in the setup: a probability density function may take values more than $1$. And without some continuity assumption on $p$ we could have $p(x) \in \{0,1\}$ for all $x$, so $p(x)(1-p(x)) = 0$ always. $\endgroup$ – Mark Wildon Jan 12 at 8:58
  • 1
    $\begingroup$ I guess that density is a misleading word here: $p(x) $ is a measure of atom $\{x\} $. $\endgroup$ – Fedor Petrov Jan 12 at 9:35
  • 1
    $\begingroup$ Ah: I see. I somehow misread 'finite support' as 'compact support'. I'm going to edit the question to make it clear that the density is discrete. $\endgroup$ – Mark Wildon Jan 12 at 9:40
9
$\begingroup$

Denote $f(p)=p(1-p)$, $H(p)=-p\log p$, let $p_1,\dots,p_n$ denote all positive probabilities of our distribution, then $\sum p_i=1$, finally denote $s=\sum_i f(p_i)$. Then we need to prove the inequality $$ \sum_i H(f(p_i)/s)\geqslant \sum_i H(p_i). $$ Since $H$ is concave, it suffices to prove that the multiset $\{p_1,\dots,p_n\}$ majorizes the multiset $\{f(p_1)/s,f(p_2)/s,\dots,f(p_n)/s\}$. We may assume that $p_1\geqslant p_2\geqslant \ldots\geqslant p_n$, it implies $f(p_1)\geqslant f(p_2)\geqslant \ldots\geqslant f(p_n)$, since $f(p_i)-f(p_j)=(p_i-p_j)(1-p_i-p_j)$, and $1-p_i-p_j\geqslant 0$ for $i\ne j$. Therefore we need to check for every $k$ that $$\frac{p_1+\dots+p_k}{p_1+\dots+p_n}=p_1+\dots+p_k\geqslant \frac{f(p_1)+\dots+f(p_k)}s,$$ that is equivalent (by multiplying through by the denominators on the far left and far right) to $$(p_1+\dots+p_k)(p_{k+1}^2+\dots+p_n^2)\leqslant (p_1^2+\dots+p_k^2)(p_{k+1}+\dots+p_n),$$ which is a sum of obvious inequalities $p_ip_j^2\leqslant p_i^2 p_j$ over all pairs $i\leqslant k<j$.

$\endgroup$
  • $\begingroup$ thanks a lot. Is there a theory about such probability transform? $\endgroup$ – sunxd Jan 16 at 22:43
  • $\begingroup$ @sunxd no idea about such theory $\endgroup$ – Fedor Petrov Jan 17 at 14:47

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.