Approximate $\log \mathbb E_P[\exp(th(x)]$ for a function $h$ which is lipschitz and has finite moments of order 1 and 2 w.r.t $P$ Let $P$ be a probability measure on a space  $\mathcal X$ and  $h: \mathcal X \rightarrow \mathbb R$ is measurable function with finite moments of order 1 and 2. I'm interested in approximating the log MGF $t \mapsto \log \mathbb E_P[\exp(th(x)]$.
Now, using a simple Taylor expansion, one can carelessly write
$$
\log \mathbb E_P[\exp(th(x)] = t\mathbb E_P[h(x)] + \frac{t^2}{2} \operatorname{Var}_P[h(x)] + o(t^2).
$$
Question


*

*Is the above representation formally correct ? If what are the formal bits missing ?

*If $\mathcal X$ has metric structure and $h$ is $M$-Lipschitz, does anything change ? That is can more be said ?


*

*For example if $P$ is a 1-subgaussian distribution and $h$ is Lipschitz, can anything more be said about $\log \mathbb E_P[\exp(th(x)]$. For example, are interesting bounds on $\log \mathbb E_P[\exp(th(x)]$ is this case ? My rough guess is that $\log \mathbb E_P[\exp(th(x)] \le M^2t^2 / 2$, but I'm not sure.


 A: To simplify notation, let $X$ be a random variable whose probability distribution is $P$, and then let $Y:=h(X)$. Your question is then is whether 
$$\ln E e^{tY}=t\,EY+\frac{t^2}2\,Var\, Y+o(t^2),$$
apparently for $t\to0$. 
Clearly, for this question to have meaning, we have to assume that the values $M(t):=E e^{tY}$ of the moment generating function (mgf) $M$ of $Y$ are finite for all $t$ in an open neighborhood $V$ of $0$. 
But then, as is well known, the mgf $M$ has derivatives $M^{(k)}$ of all orders $k$ on $V$ (actually, $M$ is even real-analytic on $V$), and $M^{(k)}(0)=EY^k$ for $k=0,1,\dots$. This follows immediately from a standard rule of differentiation of an integral with respect to a parameter; see e.g. Lemma 2.4. So, the function $L:=\ln M$ also has derivatives $L^{(k)}$ of all orders $k$ on $V$, with $L'(0)=EY$ and $L''(0)=Var\, Y$. So, by Maclaurin's expansion, your desired result follows. 
In particular, if $X$ is subgaussian and $h$ is $c$-Lipschitz, then $Y=h(X)$ is also subgaussian, since $|Y|\le|h(0)|+c|X|$, so that the mgf $M$ of $Y$ is finite everywhere on $\mathbb R$, and hence what is stated above applies.  
