# Uniform Convergence of Moment Generating Function

In the article, "The Empirical Moment Generating Function" by Csörgö, the author defines the empirical moment generating function for a sample of $n$ variables $X_1,X_2, \dots, X_n$ as: $$\begin{equation} M_n (t) = \frac{1}{n} \sum\limits_{i = 1}^{n} e^{t X_i} = \int\limits_{-\infty}^{\infty} e^{tx} dF_n (x), \end{equation}$$ where $F_n (x)$ denotes the empirical distribution function. He then states, without proof, the following proposition:

Let $M(t)$ be the moment generating function of $X$ and assume that $M$ is defined for all $t$ in a non-degenerate interval $J$ then: \begin{equation} \sup_{t \hspace{1mm} \in \hspace{1mm} J} | M_n (t) - M(t) | \to 0, \quad \textrm{as } \quad n \to \infty. \end{equation}

He establishes that the proof can be done by noticing that: \begin{equation} M_n(t) - M(t) = \int\limits_{-\infty}^{\infty} e^{tx} d\big(F_n (x) - F(x)\big), \end{equation} and then dividing the integral "into two parts $\int\limits_{|x| > A} + \int\limits_{|x| \leq A}$ and making use of the Glivenko–Cantelli theorem and another classical result, Dini's theorem" ($A$ is never specified in the article).

I am not interested in other proofs of this (such as the one in p. 459 here that uses convexity) but in understanding how Csörgö did it. Could someone please provide a more detailed guideline as how Csörgö's proof would go?

REMARK

So the first thing is that your definition of $M_{n}(t)$ needs to be $\frac{1}{n}\sum e^{tX_i}$. Now Glivenko-Cantelli tells us that $F_n$ converges uniformly to $F$. Fix $A$. Now integration by parts for the Lebesgue-Stieltjes integral gives us that $$\int_{-A}^{A} e^{tx}d(F_n(x)-F(x))=e^{tA}(F_n(A)-F(A))-e^{-tA}(F_n(-A)-F(-A))-t\cdot\int_{-A}^{A} (F_n(x)-F(x))e^{tx}dx$$.

(The layout isn't the greatest: there is a $t$ before that last integral). Uniform convergence lets us make all these contributions negligible, as $t$ is bounded on some interval.

Now we have the parts outside of $A$ to deal with. My guess is that this is where Dini's theorem comes in, letting us set $A$ big enough so the convergence is monotone (I'm not sure if that is true yet). But we can't use the integration by parts trick there, so have to be cruder.

• I have added the $1/n$ how could I forget it! Also, for large enough $A$, $F_n(x)$ is constant; as $F(x)$ is non-decreasing, $a_n=\big(F_n(x) - F(x)\big)$ is monotone. My bet is to somehow apply Dini's to $a_n$... – Rodrigo Zepeda Dec 3 '15 at 17:02
• What's needed isn't monotone in $x$, but for a given $x$ monotone in $n$. If we could apply Dini's theorem on the tails, we would have the result. – Watson Ladd Dec 3 '15 at 17:41

First of all, a few small comments: (1) $M_n(t)=M_n(t;\omega)$ is a random variable, so the claim really is that you will have uniform convergence almost surely; (2) you need to assume that $J$ is compact, or, alternatively, only claim locally uniform convergence. The convergence need not be uniform on $t\in (a,b)$ if $\int e^{bx}\, dF(x)=\infty$.

Now on to the actual question. What you quote, almost seems to suggest the following approach: suppose that $dF_n$, $dF$ are probability measures with $F_n(x)\to F(x)$ uniformly on $\mathbb R$ (this is what we get from Glivenko-Cantelli in your situation, almost surely again, and I've fixed such an $\omega$ now in my mind). Suppose also that $dF_n$ is finitely supported and $\int e^{tx}\, dF(x)<\infty$ for $t\in [a,b]$. Then we hope that $M_n(t)\to M(t)$ uniformly on $a\le t\le b$.

However, this statement is obviously false because $dF_n$ could give small but positive weight $w$ to a huge point $x$, in a such a way that $we^{tx}$ is not small.

The bottom line is that something else has to enter; you have to use some other probabilistic tool that makes use of the specifics of the situation. Taking Watson's answer into account, we essentially must make sure that $\int_A^{\infty} e^{bx}\, dF_n(x)$ stays small uniformly in $n$ for big enough $A$. To see this, the argument from the paper you linked is convenient: by the strong low of large numbers, $M_n(b)\to M(b)$ (with prob $1$, and I'll again fix such an $\omega$), but almost all contributions to $M(b)$ come from $(-A,A)$ if we take $A$ large enough, and (as we know from Watson's answer) we get almost the same answer from $\int_{-A}^A e^{bx}\, dF_n(x)$, so very little can come from $x>A$.