Approximate $\log \mathbb E_P[\exp(th(x)]$ for a function $h$ which is lipschitz and has finite moments of order 1 and 2 w.r.t $P$

Let $$P$$ be a probability measure on a space $$\mathcal X$$ and $$h: \mathcal X \rightarrow \mathbb R$$ is measurable function with finite moments of order 1 and 2. I'm interested in approximating the log MGF $$t \mapsto \log \mathbb E_P[\exp(th(x)]$$.

Now, using a simple Taylor expansion, one can carelessly write

$$\log \mathbb E_P[\exp(th(x)] = t\mathbb E_P[h(x)] + \frac{t^2}{2} \operatorname{Var}_P[h(x)] + o(t^2).$$

Question

• Is the above representation formally correct ? If what are the formal bits missing ?

• If $$\mathcal X$$ has metric structure and $$h$$ is $$M$$-Lipschitz, does anything change ? That is can more be said ?

• For example if $$P$$ is a 1-subgaussian distribution and $$h$$ is Lipschitz, can anything more be said about $$\log \mathbb E_P[\exp(th(x)]$$. For example, are interesting bounds on $$\log \mathbb E_P[\exp(th(x)]$$ is this case ? My rough guess is that $$\log \mathbb E_P[\exp(th(x)] \le M^2t^2 / 2$$, but I'm not sure.

To simplify notation, let $$X$$ be a random variable whose probability distribution is $$P$$, and then let $$Y:=h(X)$$. Your question is then is whether $$\ln E e^{tY}=t\,EY+\frac{t^2}2\,Var\, Y+o(t^2),$$ apparently for $$t\to0$$.
Clearly, for this question to have meaning, we have to assume that the values $$M(t):=E e^{tY}$$ of the moment generating function (mgf) $$M$$ of $$Y$$ are finite for all $$t$$ in an open neighborhood $$V$$ of $$0$$.
But then, as is well known, the mgf $$M$$ has derivatives $$M^{(k)}$$ of all orders $$k$$ on $$V$$ (actually, $$M$$ is even real-analytic on $$V$$), and $$M^{(k)}(0)=EY^k$$ for $$k=0,1,\dots$$. This follows immediately from a standard rule of differentiation of an integral with respect to a parameter; see e.g. Lemma 2.4. So, the function $$L:=\ln M$$ also has derivatives $$L^{(k)}$$ of all orders $$k$$ on $$V$$, with $$L'(0)=EY$$ and $$L''(0)=Var\, Y$$. So, by Maclaurin's expansion, your desired result follows.
In particular, if $$X$$ is subgaussian and $$h$$ is $$c$$-Lipschitz, then $$Y=h(X)$$ is also subgaussian, since $$|Y|\le|h(0)|+c|X|$$, so that the mgf $$M$$ of $$Y$$ is finite everywhere on $$\mathbb R$$, and hence what is stated above applies.