8
$\begingroup$

This is a cross-post to a yet unanswered question in Math StackExchange

https://math.stackexchange.com/questions/3906767/probability-of-a-deviation-when-jensen-s-inequality-is-almost-tight

Let $X>0$ be a random variable. Suppose that we knew that for some $\epsilon \geq 0$, \begin{eqnarray} \log(E[X]) \leq E[\log(X)] + \epsilon \tag{1} \label{eq:primary} \end{eqnarray} The question is: if $\epsilon$ is small, can we find a good bound for \begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) \end{eqnarray*} for a given $\eta > 0$. One bound can be obtained this way: \begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) &=& P\left( X > \exp(E[\log(X)] + \eta) \right) \\ & \leq & E[X] / \exp(E[\log(X)] + \eta) \\ & = & \exp( \log E[X] - E[\log(X)] - \eta ) \\ & \leq & \exp(\epsilon - \eta) \end{eqnarray*} where the first inequality follows from Markov’s inequality. This seems like a good bound due to the exponential decay with $\eta$, but upon closer examination it appears that it can be significantly improved. If we have $\epsilon = 0$, then this bounds gives \begin{eqnarray} P\left( \log(X) > E[\log(X)] + \eta \right) & \leq & \exp(-\eta) \tag{2} \label{eq:good_but_not_best} \end{eqnarray} However, from Jensen's inequality applied to (\ref{eq:primary}) with $\epsilon = 0$ we obtain $\log(E[X]) = E[\log(X)]$ and therefore $X$ is a constant almost everywhere. As a consequence, for any $\eta>0$, \begin{eqnarray*} P\left( \log(X) > E[\log(X)] + \eta \right) = 0. \end{eqnarray*} which is (of course) infinitely better than (\ref{eq:good_but_not_best}).

It would appear that a better bound should decay to zero as $\epsilon$ decays, and ideally preserve the exponential decay with $\eta$. Any suggestions?

(I am aware a version of this question has been asked previously Quantitative Version of Jensen's Inequality?)

$\endgroup$

1 Answer 1

3
$\begingroup$

$\newcommand\ep\epsilon $Let $u:=\eta>0$, so that the probability in question is $P(\ln X>E\ln X+u)$. Note that this probability will not change if we replace there $X$ by $tX$ for any real $t>0$. So, without loss of generality \begin{equation*} E\ln X=0, \tag{-1} \end{equation*} and hence your condition (1) can be rewritten as \begin{equation*} EX\le e^\ep, \tag{0} \end{equation*} and then the probability in question simplifies to \begin{equation*} P(X>v), \end{equation*} where \begin{equation*} v:=e^u>1. \end{equation*} Take now any $z\in(0,v)$ and for all real $x>0$ let
\begin{equation*} g(x):=ax-b\ln x+c, \end{equation*} where \begin{equation*} a:=a(z):=\frac{1/v}{h(r)},\quad b:=b(z):=az,\quad c:=c(z):=az\ln\frac ze, \end{equation*} \begin{equation*} h(r):=1-r+r\ln r,\quad r:=z/v\in(0,1). \end{equation*} Note that the function $h$ is decreasing on $(0,1)$, with $h(1-)=0$. So, $h>0$ on $(0,1)$ and hence $a>0$ and $b>0$. So, the function $g$ is convex on $(0,\infty)$. Moreover, \begin{equation*} g(z)=g'(z)=0, \quad g(v)=1. \end{equation*} It follows that $g(x)\ge1(x>v)$ for all real $x>0$ and hence, in view of (-1) and (0),
\begin{equation*} P(X>v)\le Eg(X)=a\,EX+c\le ae^\ep+c. \tag{1} \end{equation*} The latter expression, $ae^\ep+c$, in (1) can now be minimized in $z\in(0,v)$, with the minimizer expressed in terms of Lambert's $W$ function.

The suboptimal but simple choice $z=1$ in (1) yields \begin{equation*} P(\ln X>E\ln X+u)=P(X>v)\le\frac{e^\ep-1}{v-1-\ln v} \end{equation*} and hence \begin{equation*} P(\ln X>E\ln X+u)\le B_\ep(u):=\min\Big(1,\frac{e^\ep-1}{e^u-1-u}\Big). \end{equation*} The simple upper bound $B_\ep(u)$ has both of the desired properties:

(i) for each real $u>0$ \begin{equation*} B_\ep(u)\underset{\ep\downarrow0}\longrightarrow0; \end{equation*}

(ii) uniformly over all $\ep\in(0,1)$ (say) \begin{equation*} B_\ep(u)=O(e^{-u}) \end{equation*} as $u\to\infty$.

$\endgroup$
2
  • $\begingroup$ Thank you! The idea is neat. I believe you need some adjustments in the definition of $h(r)$ with $h(r) = 1-r+r \log r$ to ensure that $g(v)=1$. $\endgroup$
    – Luis L.
    Dec 2, 2020 at 5:07
  • $\begingroup$ @LuisL. : Indeed, the correct definition of $h$ is $h(r):=1-r+r\ln r$. This is now fixed. $\endgroup$ Dec 2, 2020 at 5:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.