# Shannon entropy of $p(x)(1-p(x))$ is no less than entropy of $p(x)$

If $$p(x)$$ is a discrete probabilistic density function, one could construct another discrete probabilistic density function proportional to $$p(x)[1-p(x)]$$ with a corresponding partition function to make the new function sum to $$1$$. (Rule out the degenerate case where $$p(x_0) = 1$$ for a unique $$x_0$$.)

It seems that the new constructed p.d.f has a bigger or equal entropy than $$p(x)$$. How to prove that? Suggestions are welcomed for any references or direct solution.

• There is something wrong in the setup: a probability density function may take values more than $1$. And without some continuity assumption on $p$ we could have $p(x) \in \{0,1\}$ for all $x$, so $p(x)(1-p(x)) = 0$ always. – Mark Wildon Jan 12 at 8:58
• I guess that density is a misleading word here: $p(x)$ is a measure of atom $\{x\}$. – Fedor Petrov Jan 12 at 9:35
• Ah: I see. I somehow misread 'finite support' as 'compact support'. I'm going to edit the question to make it clear that the density is discrete. – Mark Wildon Jan 12 at 9:40

Denote $$f(p)=p(1-p)$$, $$H(p)=-p\log p$$, let $$p_1,\dots,p_n$$ denote all positive probabilities of our distribution, then $$\sum p_i=1$$, finally denote $$s=\sum_i f(p_i)$$. Then we need to prove the inequality $$\sum_i H(f(p_i)/s)\geqslant \sum_i H(p_i).$$ Since $$H$$ is concave, it suffices to prove that the multiset $$\{p_1,\dots,p_n\}$$ majorizes the multiset $$\{f(p_1)/s,f(p_2)/s,\dots,f(p_n)/s\}$$. We may assume that $$p_1\geqslant p_2\geqslant \ldots\geqslant p_n$$, it implies $$f(p_1)\geqslant f(p_2)\geqslant \ldots\geqslant f(p_n)$$, since $$f(p_i)-f(p_j)=(p_i-p_j)(1-p_i-p_j)$$, and $$1-p_i-p_j\geqslant 0$$ for $$i\ne j$$. Therefore we need to check for every $$k$$ that $$\frac{p_1+\dots+p_k}{p_1+\dots+p_n}=p_1+\dots+p_k\geqslant \frac{f(p_1)+\dots+f(p_k)}s,$$ that is equivalent (by multiplying through by the denominators on the far left and far right) to $$(p_1+\dots+p_k)(p_{k+1}^2+\dots+p_n^2)\leqslant (p_1^2+\dots+p_k^2)(p_{k+1}+\dots+p_n),$$ which is a sum of obvious inequalities $$p_ip_j^2\leqslant p_i^2 p_j$$ over all pairs $$i\leqslant k.