Let $P$ be a probability measure on a space $\mathcal X$ and $h: \mathcal X \rightarrow \mathbb R$ is measurable function with finite moments of order 1 and 2. I'm interested in approximating the log MGF $t \mapsto \log \mathbb E_P[\exp(th(x)]$.

Now, using a simple Taylor expansion, one can carelessly write

$$ \log \mathbb E_P[\exp(th(x)] = t\mathbb E_P[h(x)] + \frac{t^2}{2} \operatorname{Var}_P[h(x)] + o(t^2). $$


  • Is the above representation formally correct ? If what are the formal bits missing ?

  • If $\mathcal X$ has metric structure and $h$ is $M$-Lipschitz, does anything change ? That is can more be said ?

    • For example if $P$ is a 1-subgaussian distribution and $h$ is Lipschitz, can anything more be said about $\log \mathbb E_P[\exp(th(x)]$. For example, are interesting bounds on $\log \mathbb E_P[\exp(th(x)]$ is this case ? My rough guess is that $\log \mathbb E_P[\exp(th(x)] \le M^2t^2 / 2$, but I'm not sure.

To simplify notation, let $X$ be a random variable whose probability distribution is $P$, and then let $Y:=h(X)$. Your question is then is whether $$\ln E e^{tY}=t\,EY+\frac{t^2}2\,Var\, Y+o(t^2),$$ apparently for $t\to0$.

Clearly, for this question to have meaning, we have to assume that the values $M(t):=E e^{tY}$ of the moment generating function (mgf) $M$ of $Y$ are finite for all $t$ in an open neighborhood $V$ of $0$.

But then, as is well known, the mgf $M$ has derivatives $M^{(k)}$ of all orders $k$ on $V$ (actually, $M$ is even real-analytic on $V$), and $M^{(k)}(0)=EY^k$ for $k=0,1,\dots$. This follows immediately from a standard rule of differentiation of an integral with respect to a parameter; see e.g. Lemma 2.4. So, the function $L:=\ln M$ also has derivatives $L^{(k)}$ of all orders $k$ on $V$, with $L'(0)=EY$ and $L''(0)=Var\, Y$. So, by Maclaurin's expansion, your desired result follows.

In particular, if $X$ is subgaussian and $h$ is $c$-Lipschitz, then $Y=h(X)$ is also subgaussian, since $|Y|\le|h(0)|+c|X|$, so that the mgf $M$ of $Y$ is finite everywhere on $\mathbb R$, and hence what is stated above applies.

  • $\begingroup$ Great answer. Thanks! $\endgroup$ – dohmatob Feb 13 at 15:10

Your Answer

By clicking "Post Your Answer", you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.