8
$\begingroup$

What is the motivation of defining the moment generating function of a random variable $X$ as: $E[e^{tX}]$? I know that one can obtain the mean, second moment, etc.. after computing it. But what was the intuition in using $e^{x}$? Is it because its one-to-one and always increasing?

$\endgroup$

4 Answers 4

9
$\begingroup$

If X and Y are independent then $E[e^{t(X+Y)}] = E[e^{tX}] E[e^{tY}]$, so convolution corresponds to multiplication of the mgf's. Another reason: the moment generating function is actually a Fourier transform.

Now suppose $X_i$ are i.i.d. with zero mean, and define $Y_n = \sum_{i=1}^n X_i/\sqrt{n}$. Define $\phi(t) = E[e^{tX_1}]$. Then $E[e^{tY_n}] = \phi(t/\sqrt{n})^n$. Under reasonable assumptions, $\phi(t) = 1 + V[X_1]t^2/2 + O(t^3)$, and so $E[e^{tY_n}] = (1 + V[X_1]t^2/2n + O(t^3)/n^{1.5})^n \longrightarrow e^{V[X_1]t^2/2}$, and we get the central limit theorem (by continuity of the Fourier transform).

$\endgroup$
1
  • 7
    $\begingroup$ Actually, it's a Laplace transform. The difference is important: it's possible to find $X_1$ such that $\phi(t)$ diverges except for $t=0$. But the Fourier transform $E(e^{itX})$ is defined for every real-valued random variable and every real $t$. $\endgroup$ Aug 1, 2010 at 3:30
6
$\begingroup$

The goal is to to put all the moments in one package. Since $$ e^{tx} = \sum \frac{x^n}{n!} t^n $$ the coefficients of $t^n$ in $E(e^{tx})$ are (scaled) moments. In other contexts we can use $$ (1-xt)^{-1} = \sum x^n t^n $$ in place of $e^{tx}$. This gives more or less what engineers call the "z-transform" and in combinatorics it is known as "ordinary generating function". Using the exponential has the happy advantage that convolution of random variables translates to product of moment generating functions.

$\endgroup$
1
  • 1
    $\begingroup$ It might be worth mentioning that many of the PDFs that arise in applications form exponential families. So computing the expected value of $\exp(tx)$ requires adding a simple $tx$ term into an already existing exponential and you find you have an integral that is no more difficult than proving the PDF itself integrates to 1. This makes the computing the MGF surprisingly tractable. Typically this is not the case when dealing with $(1-xt)^{-1}$. The former is also more likely to converge. $\endgroup$
    – Dan Piponi
    Feb 20, 2018 at 19:22
5
$\begingroup$

Take the definition of "generating function" for a sequence. Do it where the sequence is the sequence consisting of the moments of $X$. That's it.

$\endgroup$
3
$\begingroup$

As you suggest, the fact that $e^x$ is increasing is a useful property here. In particular, that lets you in some cases to apply Markov's/Chebyshev's inequality to the random variable $e^{tX}$ in order to get exponentially decaying bounds on the tails of $X$; see e.g. Chernoff bounds. In principle any other positive increasing function could be used in the same way, but $e^x$ is a particularly useful choice because it is especially well suited to studying sums of independent random variables, as noted already in Yuval's answer.

$\endgroup$
2
  • 1
    $\begingroup$ I think this is an answer to "What is the use of moment-generating functions", and not to "Where does the definition of moment-generating functions come from?" $\endgroup$
    – shreevatsa
    Aug 1, 2010 at 11:56
  • 1
    $\begingroup$ Fair enough. I'll try to turn this into an answer to the question that was actually asked. $\endgroup$ Aug 1, 2010 at 13:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.