Probability of a deviation when Jensen’s inequality is almost tight

Question

This is a cross-post to a yet unanswered question in Math StackExchange

https://math.stackexchange.com/questions/3906767/probability-of-a-deviation-when-jensen-s-inequality-is-almost-tight

Let $X>0$ be a random variable. Suppose that we knew that for some $\epsilon \geq 0$, \begin{eqnarray} \log(E[X]) \leq E[\log(X)] + \epsilon \tag{1} \label{eq:primary} \end{eqnarray} The question is: if $\epsilon$ is small, can we find a good bound for \begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) \end{eqnarray*} for a given $\eta > 0$. One bound can be obtained this way: \begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) &=& P\left( X > \exp(E[\log(X)] + \eta) \right) \\ & \leq & E[X] / \exp(E[\log(X)] + \eta) \\ & = & \exp( \log E[X] - E[\log(X)] - \eta ) \\ & \leq & \exp(\epsilon - \eta) \end{eqnarray*} where the first inequality follows from Markov’s inequality. This seems like a good bound due to the exponential decay with $\eta$, but upon closer examination it appears that it can be significantly improved. If we have $\epsilon = 0$, then this bounds gives \begin{eqnarray} P\left( \log(X) > E[\log(X)] + \eta \right) & \leq & \exp(-\eta) \tag{2} \label{eq:good_but_not_best} \end{eqnarray} However, from Jensen's inequality applied to (\ref{eq:primary}) with $\epsilon = 0$ we obtain $\log(E[X]) = E[\log(X)]$ and therefore $X$ is a constant almost everywhere. As a consequence, for any $\eta>0$, \begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) = 0. \end{eqnarray*} which is (of course) infinitely better than (\ref{eq:good_but_not_best}).

It would appear that a better bound should decay to zero as $\epsilon$ decays, and ideally preserve the exponential decay with $\eta$. Any suggestions?

(I am aware a version of this question has been asked previously Quantitative Version of Jensen's Inequality?)

Answer 1

$\newcommand\ep\epsilon $Let $u:=\eta>0$, so that the probability in question is $P(\ln X>E\ln X+u)$. Note that this probability will not change if we replace there $X$ by $tX$ for any real $t>0$. So, without loss of generality \begin{equation*} E\ln X=0, \tag{-1} \end{equation*} and hence your condition (1) can be rewritten as \begin{equation*} EX\le e^\ep, \tag{0} \end{equation*} and then the probability in question simplifies to \begin{equation*} P(X>v), \end{equation*} where \begin{equation*} v:=e^u>1. \end{equation*} Take now any $z\in(0,v)$ and for all real $x>0$ let
\begin{equation*} g(x):=ax-b\ln x+c, \end{equation*} where \begin{equation*} a:=a(z):=\frac{1/v}{h(r)},\quad b:=b(z):=az,\quad c:=c(z):=az\ln\frac ze, \end{equation*} \begin{equation*} h(r):=1-r+r\ln r,\quad r:=z/v\in(0,1). \end{equation*} Note that the function $h$ is decreasing on $(0,1)$, with $h(1-)=0$. So, $h>0$ on $(0,1)$ and hence $a>0$ and $b>0$. So, the function $g$ is convex on $(0,\infty)$. Moreover, \begin{equation*} g(z)=g'(z)=0, \quad g(v)=1. \end{equation*} It follows that $g(x)\ge1(x>v)$ for all real $x>0$ and hence, in view of (-1) and (0),
\begin{equation*} P(X>v)\le Eg(X)=a\,EX+c\le ae^\ep+c. \tag{1} \end{equation*} The latter expression, $ae^\ep+c$, in (1) can now be minimized in $z\in(0,v)$, with the minimizer expressed in terms of Lambert's $W$ function.

The suboptimal but simple choice $z=1$ in (1) yields \begin{equation*} P(\ln X>E\ln X+u)=P(X>v)\le\frac{e^\ep-1}{v-1-\ln v} \end{equation*} and hence \begin{equation*} P(\ln X>E\ln X+u)\le B_\ep(u):=\min\Big(1,\frac{e^\ep-1}{e^u-1-u}\Big). \end{equation*} The simple upper bound $B_\ep(u)$ has both of the desired properties:

(i) for each real $u>0$ \begin{equation*} B_\ep(u)\underset{\ep\downarrow0}\longrightarrow0; \end{equation*}

(ii) uniformly over all $\ep\in(0,1)$ (say) \begin{equation*} B_\ep(u)=O(e^{-u}) \end{equation*} as $u\to\infty$.