In the article, "The Empirical Moment Generating Function" by Csörgö, the author defines the empirical moment generating function for a sample of $n$ variables $X_1,X_2, \dots, X_n$ as: $$ \begin{equation} M_n (t) = \frac{1}{n} \sum\limits_{i = 1}^{n} e^{t X_i} = \int\limits_{-\infty}^{\infty} e^{tx} dF_n (x), \end{equation} $$ where $F_n (x)$ denotes the empirical distribution function. He then states, without proof, the following proposition:

Let $M(t)$ be the moment generating function of $X$ and assume that $M$ is defined for all $t$ in a non-degenerate interval $J$ then: \begin{equation} \sup_{t \hspace{1mm} \in \hspace{1mm} J} | M_n (t) - M(t) | \to 0, \quad \textrm{as } \quad n \to \infty. \end{equation}

He establishes that the proof can be done by noticing that: \begin{equation} M_n(t) - M(t) = \int\limits_{-\infty}^{\infty} e^{tx} d\big(F_n (x) - F(x)\big), \end{equation} and then dividing the integral "into two parts $\int\limits_{|x| > A} + \int\limits_{|x| \leq A}$ and making use of the Glivenko–Cantelli theorem and another classical result, Dini's theorem" ($A$ is never specified in the article).

I am not interested in other proofs of this (such as the one in p. 459 here that uses convexity) but in understanding how Csörgö did it. Could someone please provide a more detailed guideline as how Csörgö's proof would go?


I asked this question first in Math StackExchange but failed to receive an answer. I hope it is adequate for here.


2 Answers 2


So the first thing is that your definition of $M_{n}(t)$ needs to be $\frac{1}{n}\sum e^{tX_i}$. Now Glivenko-Cantelli tells us that $F_n$ converges uniformly to $F$. Fix $A$. Now integration by parts for the Lebesgue-Stieltjes integral gives us that $$\int_{-A}^{A} e^{tx}d(F_n(x)-F(x))=e^{tA}(F_n(A)-F(A))-e^{-tA}(F_n(-A)-F(-A))-t\cdot\int_{-A}^{A} (F_n(x)-F(x))e^{tx}dx$$.

(The layout isn't the greatest: there is a $t$ before that last integral). Uniform convergence lets us make all these contributions negligible, as $t$ is bounded on some interval.

Now we have the parts outside of $A$ to deal with. My guess is that this is where Dini's theorem comes in, letting us set $A$ big enough so the convergence is monotone (I'm not sure if that is true yet). But we can't use the integration by parts trick there, so have to be cruder.

  • 1
    $\begingroup$ I have added the $1/n$ how could I forget it! Also, for large enough $A$, $F_n(x)$ is constant; as $F(x)$ is non-decreasing, $a_n=\big(F_n(x) - F(x)\big)$ is monotone. My bet is to somehow apply Dini's to $a_n$... $\endgroup$ Dec 3, 2015 at 17:02
  • $\begingroup$ What's needed isn't monotone in $x$, but for a given $x$ monotone in $n$. If we could apply Dini's theorem on the tails, we would have the result. $\endgroup$ Dec 3, 2015 at 17:41

First of all, a few small comments: (1) $M_n(t)=M_n(t;\omega)$ is a random variable, so the claim really is that you will have uniform convergence almost surely; (2) you need to assume that $J$ is compact, or, alternatively, only claim locally uniform convergence. The convergence need not be uniform on $t\in (a,b)$ if $\int e^{bx}\, dF(x)=\infty$.

Now on to the actual question. What you quote, almost seems to suggest the following approach: suppose that $dF_n$, $dF$ are probability measures with $F_n(x)\to F(x)$ uniformly on $\mathbb R$ (this is what we get from Glivenko-Cantelli in your situation, almost surely again, and I've fixed such an $\omega$ now in my mind). Suppose also that $dF_n$ is finitely supported and $\int e^{tx}\, dF(x)<\infty$ for $t\in [a,b]$. Then we hope that $M_n(t)\to M(t)$ uniformly on $a\le t\le b$.

However, this statement is obviously false because $dF_n$ could give small but positive weight $w$ to a huge point $x$, in a such a way that $we^{tx}$ is not small.

The bottom line is that something else has to enter; you have to use some other probabilistic tool that makes use of the specifics of the situation. Taking Watson's answer into account, we essentially must make sure that $\int_A^{\infty} e^{bx}\, dF_n(x)$ stays small uniformly in $n$ for big enough $A$. To see this, the argument from the paper you linked is convenient: by the strong low of large numbers, $M_n(b)\to M(b)$ (with prob $1$, and I'll again fix such an $\omega$), but almost all contributions to $M(b)$ come from $(-A,A)$ if we take $A$ large enough, and (as we know from Watson's answer) we get almost the same answer from $\int_{-A}^A e^{bx}\, dF_n(x)$, so very little can come from $x>A$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.