3
$\begingroup$

Let $ f : [0,1] \to \mathbb{R} $ be a function satisfying: 1.) $ |f(x)| \leqslant a $ for some $ a < 1 $, and 2.) $ \int_0^1 f(x) {\mathrm d}x = 0 $. I would like to know whether the following inequality holds: $$ \int_0^1 \log(1+f(x)) \,{\mathrm d}x \; +\; \int_0^1 (1-f(x)) \log(1-f(x)) \, {\mathrm d} x \;\stackrel{?}{\geqslant}\; \log(1-a^3) $$ Note that $ 1 + f(x) $ and $ 1 - f(x) $ are probability densities on $ [0,1] $, so the stated inequality can also be written in the form: $$ D_{KL}(1 \parallel 1+f) - D_{KL}(1-f \parallel 1) \;\stackrel{?}{\leqslant}\; \log\frac{1}{1-a^3} $$ where $ D_{KL}(p \parallel q) = \int p(x) \log \frac{p(x)}{q(x)} {\mathrm d}x $ is the Kullback-Leibler divergence (relative entropy).

The inequality comes up in an information-theoretic setting and it "should" be true, but I failed to prove it formally. I did confirm it numerically in some special cases, e.g., when $ f(x) = a \cos(2\pi n x) $ for various $ a, n $, when $ f(x) = \pm a $ (each on one half of the domain), etc.

$\endgroup$
1

1 Answer 1

3
$\begingroup$

Yes, the inequality is true. Indeed, the inequality in question can be rewritten (or, if you prefer, generalized) as \begin{equation} Eg(Y)\ge\ln(1-a^3), \tag{10}\label{10} \end{equation} where $$g(t):=\ln(1+t)+(1-t)\ln(1-t)$$ and $Y$ is a random variable (r.v.) such that \begin{equation} P(|Y|\le a)=1\quad\text{and}\quad EY=0. \tag{20}\label{20} \end{equation} (One can take $Y:=f(X)$, where $X$ is a r.v. uniformly distributed on $[0,1]$.)

By well-known results (see e.g. this paper or this paper or Corollary 13 or formula (2.13)), without loss of generality (wlog) the distribution of the r.v. $Y$ is supported on a set $\{u,v\}\subset[-a,a]$ of cardinality $\le2$. Given such $u$ and $v$, if $a$ is now replaced by the smallest possible value of $b$ such that $\{u,v\}\subseteq[-b,b]$, the left-hand side of \eqref{10} will not change whereas the right-hand side of \eqref{10} will not decrease. So, wlog one of the values $u$ or $v$ coincides with one of the endpoints of the interval $[-a,a]$; that is, wlog either $v=-a$ or $v=a$. To cover both of these cases at once, assume wlog that the distribution of the r.v. $Y$ is supported on a set of the form $\{-a,u,a\}$ with $u\in[-a,a]$.

So then, it remains to show that
\begin{equation*} L:=pg(-a)+qg(u)+rg(a)-\ln(1-a^3)\ge0 \tag{30}\label{30} \end{equation*} given the conditions \begin{equation*} -a\le u\le a,\quad p,q,r\ge0,\quad \\ p+q+r=1,\quad p(-a)+qu+sa=0. \tag{40}\label{40} \end{equation*} Solving the latter two equations in \eqref{40} for $p$ and $r$ and substituting the resulting expressions for $p$ and $r$ into \eqref{30}, we get \begin{equation*} L=M(q,a,u):=P(a)+qR(a,u), \end{equation*} where \begin{equation*} \begin{aligned} P(a)&:=a \tanh ^{-1}(a)+\ln \left(1-a^2\right)-\ln \left(1-a^3\right) \\ Q(a,u)&:=g(u)-(1-u/2) \ln \left(1-a^2\right)-a \tanh ^{-1}(a). \end{aligned} \end{equation*} Note that $M(q,a,u)$ is affine in $q$. So, to prove \eqref{30} given \eqref{40}, it suffices to show that \begin{equation*} M(0,a,u)\ge0\quad\text{and}\quad M(1,a,u)\ge0 \tag{50}\label{50} \end{equation*} for $u\in[-a,a]$ and $a\in(0,1)$.

We have $M(0,a,u)=P(a)$, $P(0)=0=P'(0)$ and \begin{equation*} P''(a)=H(a):=\dfrac{a \left(6+10 a+2 a^2-3 a^3+2 a^4+a^5\right)}{(1-a)^2 (1+a)^2 \left(1+a+a^2\right)^2}, \tag{60}\label{60} \end{equation*} which is obviously $>0$ for $a\in(0,1)$. So, $M(0,a,u)=P(a)>0$ for $a\in(0,1)$.

Next, \begin{equation*} M(1,a,u)=F(a):=F(a,u):=g(u)-\ln \left(1-a^3\right) \\ +\frac u2\, \ln \left(1-a^2\right), \end{equation*} \begin{equation*} F'(a)\frac{(1 - a) (1 + a) (1 + a + a^2)}a =3 a (1 + a) - (1+ a +a^2) u \\ \ge3 a (1 + a) - (1+ a +a^2) a =a (2 + 2 a - a^2)>0. \end{equation*} So, $F$ is increasing in $a$ and hence $M(1,a,u)=F(a,u)\ge F(|u|,u)$, because $|u|\le a$. If now $u\in[0,1)$, then $F(|u|,u)=F(u,u)=P(u)\ge0$, by what was shown in the previous paragraph. Finally, if $u\in(-1,0)$, then $F(|u|,u)=F(-u,u)=h(u):=g(u)+\frac u2\, \ln(1 - u^2) - \ln(1 + u^3)$, $h(0)=0=h'(0)$, $h''(u)=H(|u|)>0$ -- cf. \eqref{60}; so, $M(1,a,u)=F(a,u)\ge F(|u|,u)=F(-u,u)=h(u)\ge0$.

Thus, \eqref{50} is proved. $\quad\Box$

$\endgroup$
1
  • $\begingroup$ Fantastic, thank you! $\endgroup$
    – aleph
    Commented Jun 28, 2023 at 11:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .