2
$\begingroup$

Let $X$ be a nonnegative random variable such that $\mathbf{E} \left[ \exp X \right] < \infty$. For $\theta \leqslant 1$, an appropriate application of Jensen's inequality, yields that

\begin{align} \mathbf{E} \left[ \exp \left( \theta X \right) \right] \leqslant \exp \left( \theta \mathbf{E} \left[ X \right] \right) + \theta^2 \left( \mathbf{E} \left[ \exp X \right] - \exp \left( \mathbf{E} \left[ X \right] \right) \right). \end{align}

As a consequence, I can obtain the following bound for the moment-generating function of $R \left( X + X^\prime \right)$, where $X, X^\prime$ are independent copies of $X$ and $R$ is a standard Rademacher random variable, independent of $X, X^\prime$ (now for $| \theta | \leqslant 1$):

\begin{equation} \mathbf{E} \left[ \exp\left(\theta R\left(X+X^{\prime}\right)\right)\right] \leqslant \left(\cosh\left(\theta\mathbf{E}\left[X\right]\right) + \theta^{2} \left(\mathbf{E}\left[\exp X\right] - \exp\left(\mathbf{E}\left[X\right]\right)\right)\right)^{2} \\ + \sinh^{2}\left(\theta\mathbf{E}\left[X\right]\right). \end{equation}

I would like a simple upper bound on this quantity, ideally of the form $\exp\left( \frac{s}{2} \theta^2 \right)$, and with a reasonably sharp constant $s$. One can check that this is the right behaviour around $\theta = 0$, and "large" $\theta$ are not really of interest anyways (since we only go as far as $\theta = \pm 1$ anyways), and so the resulting bound should be quite usable and probably even not too loose.

I have considered taking the logarithm of this quantity, differentiating this twice with respect to $\theta$, and then trying to upper-bound this uniformly, but this seems likely to get messy. A less painful solution would thus be desirable.


Proof of initial estimate:

For $x \geqslant 0$, the function $F_\theta : x \mapsto \theta^2 \exp(x) - \exp(\theta x)$ has second derivative $F_\theta^{\prime \prime} (x) = \theta^2 ( \exp(x) - \exp(\theta x)) \geqslant 0$. It hence follows that $F_\theta$ is convex. Applying Jensen's inequality then yields that $\mathbf{E} \left[ F_\theta (X) \right] \geqslant F_\theta (\mathbf{E} \left[ X \right])$. The claimed estimate follows by rearrangement.

Proof of deduction:

Observe that

\begin{align} \mathbf{E} \left[ \exp\left(\theta R\left(X+X^{\prime}\right)\right)\right] &= \mathbf{E}_R \mathbf{E} \left[ \exp\left(\theta R\left(X+X^{\prime}\right)\right)\mid R \right] \\ &= \mathbf{E}_R \left[ \mathbf{E} \left[ \exp\left(\theta R X \right) \mid R \right]^2 \right] \\ &= \frac{1}{2} \left( \mathbf{E} \left[ \exp\left(\theta X \right) \right]^2 + \mathbf{E} \left[ \exp\left(- \theta X \right) \right]^2 \right); \end{align}

the remainder follows by applying the preceding estimates for $\mathbf{E} \left[ \exp\left(\theta X \right) \right]$ and $\mathbf{E} \left[ \exp\left(-\theta X \right) \right]$, and then making algebraic manipulations.

$\endgroup$
3
  • 1
    $\begingroup$ Is $R$ independent of $X+X'$? $\endgroup$ Oct 3, 2023 at 13:15
  • 1
    $\begingroup$ Also, can you present your proofs of your displayed inequalities (or give a link to them)? $\endgroup$ Oct 3, 2023 at 13:18
  • $\begingroup$ Re: independence, yes; re: the inequality, I will edit the post. $\endgroup$
    – πr8
    Oct 3, 2023 at 13:49

1 Answer 1

2
$\begingroup$

Your upper bound on $\mathbf{E} \left[ \exp\left(\theta R\left(X+X'\right)\right)\right]$ is $$f(a,b;t):=(\cosh at+bt^2)^2+\sinh^2 at,$$ where $$a:=EX\ge0,\quad b:=Ee^X-e^{EX}\ge0,\quad t:=\theta.$$ You want to upper-bound $f(a,b;t)$ by $e^{st^2/2}$.

Expanding $f(a,b;t)$ and $e^{st^2/2}$ into powers of $t$ and comparing the coefficients of the same powers of $t$, we see that $$s=4(a^2+b) \tag{10}\label{10}$$ will do.

Indeed, we have $$f(a,b;t)=\sum_{k=0}^\infty c_k t^{2k},\quad e^{st^2/2}=\sum_{k=0}^\infty C_k t^{2k},$$ where $$c_0:=1,\quad c_1:=2(a^2+b),\quad c_2:=\frac{2 a^4}3 + a^2 b + b^2,$$ $$c_k:=\frac{a^{2 k-2} \left(a^2 2^{2 k}+4 b k (2 k-1)\right)}{(2 k)!}\text{ for }k\ge3,$$ $$C_k:=\frac{(s/2)^k}{k!} \text{ for }k\ge0.$$

It is straightforward to check that $c_0=C_0$, $c_1=C_1$, and $c_2\le C_2/2$ given \eqref{10}. It is also easy to see that $\dfrac{c_k}{C_k}$ is decreasing in $k\ge2$ given \eqref{10}. Thus, the claim follows. $\quad\Box$

Moreover, since $c_0=C_0$ and $c_1=C_1$ given \eqref{10}, we see that the expression for $s$ in \eqref{10} is the best possible one.

$\endgroup$
1
  • $\begingroup$ Terrific, this seems to be just the right strategy for the problem - many thanks. $\endgroup$
    – πr8
    Oct 3, 2023 at 17:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.