Existence and sharpness of Bernstein-type bounds on the moment-generating function

Question

Let $X$ be a centred random variable with variance $\sigma^2$, and whose moment-generating function exists in an open neighbourhood of the origin.

Say that $X$ satisfies a 'Bernstein-type' MGF bound with parameter $b \in [ 0, \infty )$ if there holds the inequality

$$\log \left(\mathbf{E} \left[ \exp \left( t \cdot X\right) \right] \right) \leq \frac{\frac{1}{2} \cdot \sigma^2 \cdot t^2}{1 - b \cdot | t |}, \qquad t \in \left( - \frac{1}{b}, \frac{1}{b} \right),$$

where it should be emphasised that the $\sigma^2$ appearing here is exactly the variance of $X$, rather than being a free parameter.

I have two quite basic questions about this:

1. Is it clear that under the given assumptions, $X$ will necessarily satisfy some Bernstein-type MGF bound? This seems quite intuitive to me, but I have not really seen it discussed.
2. Assuming that this is the case, it is natural to ask for the minimal (infimal, really) such $b$ for a given $X$; call this $b_\star \left( X \right)$. Is much known about this $b_\star \left( X \right)$?
   • For instance, can it easily be related to some Orlicz-type norm?
   • One main utility of this $b$ in many cases is that it gives some precise information on when the CLT effectively 'kicks in' for iid sums of $X$, so perhaps it will show up in this context. Though of course, the quantity is only relevant for the quite-restrictive class of $X$ with some exponential integrability, so perhaps it wouldn't have been as interesting classically in this context.

I remark in passing that if one can take $b = 0$, then such random variables are known as 'strictly' or 'strongly' sub-Gaussian, and a few interesting things are known in this case (see e.g. papers of Bobkov), which even have a Fourier-analytic character. I don't know the works well enough to intuit whether they imply anything about the case of $b > 0$.

Answer 1

$\newcommand\si\sigma$After the clarification by the OP, my previous answer should be modified as follows.

Suppose that for some real $C$, some real $h>0$, and all $t\in[-h,h]$ we have $$M(t):=Ee^{tX}\le C \tag{1}\label{1} $$ (note that necessarily $C\ge M(0)=1$.) We will show that then $$Ee^{tX}\le1+\frac{\si^2 t^2/2}{1-b|t|}\le\exp\frac{\si^2 t^2/2}{1-b|t|} \tag{2}\label{2} $$ for $$b:=\frac{4C}{\sqrt3\,h^3\si^2} \tag{3}\label{3} $$ and all $t\in[-1/b,1/b]$ (excluding the trivial case $\si=0$). Note that $C\ge M(h)\ge 1+\si^2 h^2/2>\si^2 h^2/2$ and hence $b h>\frac2{\sqrt3}>1$, so that $b>1/h$.

Indeed, \eqref{1} implies that for $m=1,2,\dots$ $$\frac{h^{2m}}{(2m)!}\,E X^{2m}\le E\cosh hX\le C,$$ so that $$E X^{2m}\le C\frac{(2m)!}{h^{2m}}.$$ Using now the Cauchy--Schwarz inequality, for $m=1,2,\dots$ we get \begin{align*} E|X|^{2m+1}&\le\sqrt{E X^{2m}\,E X^{2m+2}} \\ &\le CC\frac{\sqrt{(2m)!(2m+2)!}}{h^{2m+1}} \\ &\le \sqrt{\frac43}\, C\frac{(2m+1)!}{h^{2m+1}}. \end{align*} So, \begin{align*} E|X|^k&\le\sqrt{\frac43}\, C\frac{k!}{h^k} \end{align*} for all $k\in\{3,4\dots\}$.

So, for $b$ as in \eqref{3} and $t\in(-1/b,1/b)$, $$Ee^{tX}=\sum_{k=0}^\infty EX^k\frac{t^k}{k!} \le1+\frac{\si^2 t^2}2+\sum_{k=3}^\infty \sqrt{\frac43}\, C\frac{k!}{h^k}\frac{|t|^k}{k!} \\ =1+\frac{\si^2 t^2}2+\sqrt{\frac43}\, C\frac{(|t|/h)^3}{1-|t|/h} \le1+\frac{\si^2 t^2/2}{1-b|t|}, \tag{4}\label{4} $$ so that \eqref{2} is proved. $\quad\Box$

One may also note that the latter inequality in \eqref{4} is exact in the sense that $$1+\frac{\si^2 t^2}2+\sqrt{\frac43}\, C\frac{(|t|/h)^3}{1-|t|/h} =1+\frac{\si^2 t^2/2}{1-b|t|}-O(t^4)$$ for $t\downarrow0$.

Answer 2

I believe that (a close analogue of) your MGF bound implies Bernstein's inequality is well-known. See for example Theorem 2.8.4 and Exercise 2.8.5 of Vershynin.

As for a "Bernstein-type Orlicz norm", this has been proposed via the Bernstein-Orlicz norm. This is the Orlicz norm associated with the (parameterized by $L>0$) family of functions

$$ \Psi_L(x) := \exp\left[\frac{\sqrt{1+2Lz}-1}{L}\right]^2-1, $$ with inverse

$$ \Psi_L^{-1}(t) = \sqrt{\log(1+t)} + \frac{L}{2}\log(1+t). $$

One then defines the Orlicz norm in the standard way, e.g. $\lVert X\rVert_{\Psi_L} = \inf\{c > 0 : \mathbb{E}[\Psi_L(|X|/c)] \leq 1\}$. The authors justify their name for this Orlicz norm via the following two results

Lemma 1: Let $\tau := \lVert Z\rVert_{\Psi_L}$. Then, for any $t > 0$, $$\tag{1}\Pr[|Z| > \tau(\sqrt{t} + \frac{Lt}{2})] \leq 2\exp(-t)$$

Lemma 2: If there are constants $\tau$ and $L$ such that, for all $t > 0$, $Z$ satisfies Eq. (1), then $\lVert Z\rVert_{\Psi_{\sqrt{3}L}} \leq \sqrt{3}\tau$.

This is to say that this Orlicz norm captures random variables that satisfy the tail-bound of Eq. (1), up to an (explicit) multiplicative constant. While this isn't want you have cited as Bernstein's inequality, it has appeared under the name previously, see for example their Theorem 1, which cites Bennett 1962. It's also worth mentioning that this norm has been generalized to the larger class of sub-Weibull random variables.

Answer 3

$\newcommand\si\sigma$Your condition on the moment generating function of $X$ implies that for such $|t|<1/b$ and $m=1,2,\dots$, $$\frac{t^{2m}}{(2m)!}\,E X^{2m}\le E\cosh tX\le\exp\frac{t^2 \si^2}{2(1-b|t|)}.$$ Taking now any real $c_1>0$ and any $c_2\in(0,1)$, and then choosing $t=\min(\frac{c_1}\si,\frac{c_2}b)$, we get $$E X^{2m}\le C(2m)!\max\Big(\frac\si{c_1},\frac{b}{c_2}\Big)^{2m},$$ with $C=\exp\frac{c_1^2}{2(1-c_2)}$. Using now the Cauchy--Schwarz inequality, for $m=1,2,\dots$ we get \begin{align*} E|X|^{2m+1}&\le\sqrt{E X^{2m}\,E X^{2m+2}} \\ &\le C\sqrt{(2m)!(2m+2)!}\max\Big(\frac\si{c_1},\frac{b}{c_2}\Big)^{2m+1} \\ &\le \sqrt{\frac43}\, C(2m+1)!\max\Big(\frac\si{c_1},\frac{b}{c_2}\Big)^{2m+1}. \end{align*} So, we have the Bernstein-type condition \begin{align*} E|X|^k&\le\sqrt{\frac43}\, Ck!\max\Big(\frac\si{c_1},\frac{b}{c_2}\Big)^k \end{align*} for all $k\in\{2,3,\dots\}$.

One may note here that $\max\big(\frac\si{c_1},\frac{b}{c_2}\big)$ can be made arbitrarily close to $b$ by taking $c_2$ close to $1$ and then letting $c_1=c_2\si/b$. On the other hand, your condition on the moment generating function of $X$ is implied by the Bernstein condition $|EX^k|\le\frac{k!}2\,\si^2 b^{k-2}$ on moments of $X$ for $k=3,4,\dots$.

---

This is a modified/detailed version of the this previous answer.