Motivation of Moment Generating Functions

Question

What is the motivation of defining the moment generating function of a random variable $X$ as: $E[e^{tX}]$? I know that one can obtain the mean, second moment, etc.. after computing it. But what was the intuition in using $e^{x}$? Is it because its one-to-one and always increasing?

Answer 1

If X and Y are independent then $E[e^{t(X+Y)}] = E[e^{tX}] E[e^{tY}]$, so convolution corresponds to multiplication of the mgf's. Another reason: the moment generating function is actually a Fourier transform.

Now suppose $X_i$ are i.i.d. with zero mean, and define $Y_n = \sum_{i=1}^n X_i/\sqrt{n}$. Define $\phi(t) = E[e^{tX_1}]$. Then $E[e^{tY_n}] = \phi(t/\sqrt{n})^n$. Under reasonable assumptions, $\phi(t) = 1 + V[X_1]t^2/2 + O(t^3)$, and so $E[e^{tY_n}] = (1 + V[X_1]t^2/2n + O(t^3)/n^{1.5})^n \longrightarrow e^{V[X_1]t^2/2}$, and we get the central limit theorem (by continuity of the Fourier transform).

Answer 2

The goal is to to put all the moments in one package. Since $$ e^{tx} = \sum \frac{x^n}{n!} t^n $$ the coefficients of $t^n$ in $E(e^{tx})$ are (scaled) moments. In other contexts we can use $$ (1-xt)^{-1} = \sum x^n t^n $$ in place of $e^{tx}$. This gives more or less what engineers call the "z-transform" and in combinatorics it is known as "ordinary generating function". Using the exponential has the happy advantage that convolution of random variables translates to product of moment generating functions.

Answer 3

Take the definition of "generating function" for a sequence. Do it where the sequence is the sequence consisting of the moments of $X$. That's it.

Answer 4

As you suggest, the fact that $e^x$ is increasing is a useful property here. In particular, that lets you in some cases to apply Markov's/Chebyshev's inequality to the random variable $e^{tX}$ in order to get exponentially decaying bounds on the tails of $X$; see e.g. Chernoff bounds. In principle any other positive increasing function could be used in the same way, but $e^x$ is a particularly useful choice because it is especially well suited to studying sums of independent random variables, as noted already in Yuval's answer.