# Probability of a deviation when Jensen’s inequality is almost tight

This is a cross-post to a yet unanswered question in Math StackExchange

https://math.stackexchange.com/questions/3906767/probability-of-a-deviation-when-jensen-s-inequality-is-almost-tight

Let $$X>0$$ be a random variable. Suppose that we knew that for some $$\epsilon \geq 0$$, $$\begin{eqnarray} \log(E[X]) \leq E[\log(X)] + \epsilon \tag{1} \label{eq:primary} \end{eqnarray}$$ The question is: if $$\epsilon$$ is small, can we find a good bound for $$\begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) \end{eqnarray*}$$ for a given $$\eta > 0$$. One bound can be obtained this way: $$\begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) &=& P\left( X > \exp(E[\log(X)] + \eta) \right) \\ & \leq & E[X] / \exp(E[\log(X)] + \eta) \\ & = & \exp( \log E[X] - E[\log(X)] - \eta ) \\ & \leq & \exp(\epsilon - \eta) \end{eqnarray*}$$ where the first inequality follows from Markov’s inequality. This seems like a good bound due to the exponential decay with $$\eta$$, but upon closer examination it appears that it can be significantly improved. If we have $$\epsilon = 0$$, then this bounds gives $$\begin{eqnarray} P\left( \log(X) > E[\log(X)] + \eta \right) & \leq & \exp(-\eta) \tag{2} \label{eq:good_but_not_best} \end{eqnarray}$$ However, from Jensen's inequality applied to (\ref{eq:primary}) with $$\epsilon = 0$$ we obtain $$\log(E[X]) = E[\log(X)]$$ and therefore $$X$$ is a constant almost everywhere. As a consequence, for any $$\eta>0$$, $$\begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) = 0. \end{eqnarray*}$$ which is (of course) infinitely better than (\ref{eq:good_but_not_best}).

It would appear that a better bound should decay to zero as $$\epsilon$$ decays, and ideally preserve the exponential decay with $$\eta$$. Any suggestions?

(I am aware a version of this question has been asked previously Quantitative Version of Jensen's Inequality?)

$$\newcommand\ep\epsilon$$Let $$u:=\eta>0$$, so that the probability in question is $$P(\ln X>E\ln X+u)$$. Note that this probability will not change if we replace there $$X$$ by $$tX$$ for any real $$t>0$$. So, without loss of generality $$\begin{equation*} E\ln X=0, \tag{-1} \end{equation*}$$ and hence your condition (1) can be rewritten as $$\begin{equation*} EX\le e^\ep, \tag{0} \end{equation*}$$ and then the probability in question simplifies to $$\begin{equation*} P(X>v), \end{equation*}$$ where $$\begin{equation*} v:=e^u>1. \end{equation*}$$ Take now any $$z\in(0,v)$$ and for all real $$x>0$$ let
$$\begin{equation*} g(x):=ax-b\ln x+c, \end{equation*}$$ where $$\begin{equation*} a:=a(z):=\frac{1/v}{h(r)},\quad b:=b(z):=az,\quad c:=c(z):=az\ln\frac ze, \end{equation*}$$ $$\begin{equation*} h(r):=1-r+r\ln r,\quad r:=z/v\in(0,1). \end{equation*}$$ Note that the function $$h$$ is decreasing on $$(0,1)$$, with $$h(1-)=0$$. So, $$h>0$$ on $$(0,1)$$ and hence $$a>0$$ and $$b>0$$. So, the function $$g$$ is convex on $$(0,\infty)$$. Moreover, $$\begin{equation*} g(z)=g'(z)=0, \quad g(v)=1. \end{equation*}$$ It follows that $$g(x)\ge1(x>v)$$ for all real $$x>0$$ and hence, in view of (-1) and (0),
$$\begin{equation*} P(X>v)\le Eg(X)=a\,EX+c\le ae^\ep+c. \tag{1} \end{equation*}$$ The latter expression, $$ae^\ep+c$$, in (1) can now be minimized in $$z\in(0,v)$$, with the minimizer expressed in terms of Lambert's $$W$$ function.
The suboptimal but simple choice $$z=1$$ in (1) yields $$\begin{equation*} P(\ln X>E\ln X+u)=P(X>v)\le\frac{e^\ep-1}{v-1-\ln v} \end{equation*}$$ and hence $$\begin{equation*} P(\ln X>E\ln X+u)\le B_\ep(u):=\min\Big(1,\frac{e^\ep-1}{e^u-1-u}\Big). \end{equation*}$$ The simple upper bound $$B_\ep(u)$$ has both of the desired properties:
(i) for each real $$u>0$$ $$\begin{equation*} B_\ep(u)\underset{\ep\downarrow0}\longrightarrow0; \end{equation*}$$
(ii) uniformly over all $$\ep\in(0,1)$$ (say) $$\begin{equation*} B_\ep(u)=O(e^{-u}) \end{equation*}$$ as $$u\to\infty$$.
• Thank you! The idea is neat. I believe you need some adjustments in the definition of $h(r)$ with $h(r) = 1-r+r \log r$ to ensure that $g(v)=1$. – Luis L. Dec 2 '20 at 5:07
• @LuisL. : Indeed, the correct definition of $h$ is $h(r):=1-r+r\ln r$. This is now fixed. – Iosif Pinelis Dec 2 '20 at 5:34