Simplified upper bounds for moment-generating function of symmetrised random variable

Question

Let $X$ be a nonnegative random variable such that $\mathbf{E} \left[ \exp X \right] < \infty$. For $\theta \leqslant 1$, an appropriate application of Jensen's inequality, yields that

\begin{align} \mathbf{E} \left[ \exp \left( \theta X \right) \right] \leqslant \exp \left( \theta \mathbf{E} \left[ X \right] \right) + \theta^2 \left( \mathbf{E} \left[ \exp X \right] - \exp \left( \mathbf{E} \left[ X \right] \right) \right). \end{align}

As a consequence, I can obtain the following bound for the moment-generating function of $R \left( X + X^\prime \right)$, where $X, X^\prime$ are independent copies of $X$ and $R$ is a standard Rademacher random variable, independent of $X, X^\prime$ (now for $| \theta | \leqslant 1$):

\begin{equation} \mathbf{E} \left[ \exp\left(\theta R\left(X+X^{\prime}\right)\right)\right] \leqslant \left(\cosh\left(\theta\mathbf{E}\left[X\right]\right) + \theta^{2} \left(\mathbf{E}\left[\exp X\right] - \exp\left(\mathbf{E}\left[X\right]\right)\right)\right)^{2} \\ + \sinh^{2}\left(\theta\mathbf{E}\left[X\right]\right). \end{equation}

I would like a simple upper bound on this quantity, ideally of the form $\exp\left( \frac{s}{2} \theta^2 \right)$, and with a reasonably sharp constant $s$. One can check that this is the right behaviour around $\theta = 0$, and "large" $\theta$ are not really of interest anyways (since we only go as far as $\theta = \pm 1$ anyways), and so the resulting bound should be quite usable and probably even not too loose.

I have considered taking the logarithm of this quantity, differentiating this twice with respect to $\theta$, and then trying to upper-bound this uniformly, but this seems likely to get messy. A less painful solution would thus be desirable.

---

Proof of initial estimate:

For $x \geqslant 0$, the function $F_\theta : x \mapsto \theta^2 \exp(x) - \exp(\theta x)$ has second derivative $F_\theta^{\prime \prime} (x) = \theta^2 ( \exp(x) - \exp(\theta x)) \geqslant 0$. It hence follows that $F_\theta$ is convex. Applying Jensen's inequality then yields that $\mathbf{E} \left[ F_\theta (X) \right] \geqslant F_\theta (\mathbf{E} \left[ X \right])$. The claimed estimate follows by rearrangement.

Proof of deduction:

Observe that

\begin{align} \mathbf{E} \left[ \exp\left(\theta R\left(X+X^{\prime}\right)\right)\right] &= \mathbf{E}_R \mathbf{E} \left[ \exp\left(\theta R\left(X+X^{\prime}\right)\right)\mid R \right] \\ &= \mathbf{E}_R \left[ \mathbf{E} \left[ \exp\left(\theta R X \right) \mid R \right]^2 \right] \\ &= \frac{1}{2} \left( \mathbf{E} \left[ \exp\left(\theta X \right) \right]^2 + \mathbf{E} \left[ \exp\left(- \theta X \right) \right]^2 \right); \end{align}

the remainder follows by applying the preceding estimates for $\mathbf{E} \left[ \exp\left(\theta X \right) \right]$ and $\mathbf{E} \left[ \exp\left(-\theta X \right) \right]$, and then making algebraic manipulations.

Answer 1

Your upper bound on $\mathbf{E} \left[ \exp\left(\theta R\left(X+X'\right)\right)\right]$ is $$f(a,b;t):=(\cosh at+bt^2)^2+\sinh^2 at,$$ where $$a:=EX\ge0,\quad b:=Ee^X-e^{EX}\ge0,\quad t:=\theta.$$ You want to upper-bound $f(a,b;t)$ by $e^{st^2/2}$.

Expanding $f(a,b;t)$ and $e^{st^2/2}$ into powers of $t$ and comparing the coefficients of the same powers of $t$, we see that $$s=4(a^2+b) \tag{10}\label{10}$$ will do.

Indeed, we have $$f(a,b;t)=\sum_{k=0}^\infty c_k t^{2k},\quad e^{st^2/2}=\sum_{k=0}^\infty C_k t^{2k},$$ where $$c_0:=1,\quad c_1:=2(a^2+b),\quad c_2:=\frac{2 a^4}3 + a^2 b + b^2,$$ $$c_k:=\frac{a^{2 k-2} \left(a^2 2^{2 k}+4 b k (2 k-1)\right)}{(2 k)!}\text{ for }k\ge3,$$ $$C_k:=\frac{(s/2)^k}{k!} \text{ for }k\ge0.$$

It is straightforward to check that $c_0=C_0$, $c_1=C_1$, and $c_2\le C_2/2$ given \eqref{10}. It is also easy to see that $\dfrac{c_k}{C_k}$ is decreasing in $k\ge2$ given \eqref{10}. Thus, the claim follows. $\quad\Box$

Moreover, since $c_0=C_0$ and $c_1=C_1$ given \eqref{10}, we see that the expression for $s$ in \eqref{10} is the best possible one.