# Higher Moments, what are they good for? [closed]

Absolutely nothing?

And now seriously - When I studied the basics of probability theory, and even in more advanced topics (random walks, stochastic processes, etc.), I always felt that the mean and the variance were used extensively (and with good reason). I have wondered what possible uses do higher moments have?

• in short - moments encode the measure. E.g. you can integrate polynomials if you know moments... – Dima Pasechnik Apr 12 '15 at 21:46
• @DimaPasechnik Well, most of the time, but not always – Yemon Choi Apr 12 '15 at 23:17
• Essentially the same question was asked and answered here: stats.stackexchange.com/questions/2893/… – Mark Meckes May 27 '15 at 19:31

Perhaps a related question is Why are matrices ubiquitous but hypermatrices rare? in the sense that higher-order (mixed) moments can be regarded as hypermatrices or tensors. For example, the second order moment of a random variable $x\in\mathbb{R}^n$ might be written $\mathbb{E}[x\otimes x]$. The third order moment is $\mathbb{E}[x\otimes x\otimes x]$, and so on.

The (relative) lack of extensive use of higher order moments may be because questions about them inherit the difficulties that come with all tensor problems (Most tensor problems are NP-hard).

A very clear example of higher order moments in use is independent component analysis (ICA), which essentially amounts to a tensor decomposition of the fourth order cumulant (see here, for example). ICA is an instance of a slightly more general problem known as blind source separation, which has wide-ranging applications.

• -1: I fail to see how the second order moment is the expectation of a tensor product, unless we are interpreting tensor products very liberally – Yemon Choi Apr 12 '15 at 23:18
• I use it in the sense of an outer product, e.g. $u\otimes v=uv^\top$. I know there is an issue with the liberal use/misuse of tensors in the data analysis literature, because I see it discussed often (e.g. here). – calculusaurus Apr 12 '15 at 23:48
• OK, I think I understand what you meant, but the original question seems to be asking about the higher-order moments of a single random variable, not the mixed moments of an $n$-tuple of random variables (and I fail to see what this has to do with NP-hard) – Yemon Choi Apr 13 '15 at 0:16

If you compute the higher moments for a random variable $X$, and these coincide with the normal distribution, you know that $X$ is also normal. This can be quite powerful when you want to show that something converges to a normal distribution in the limit. A similar method works for cumulants.

I know of two such applications:

• The Berry-Esseen theorem: A quantitative version of the central limit theorem which uses the third (centered) moment.

• Some Orlicz spaces: One can define the Orlicz spaces, which are "$L^p$-esque" spaces. The ones which fit to the function $\Psi_\alpha (x) = exp(x^\alpha)-1$ are supposed to conclude RVs which decay at least as fast as RV whose density is $C\cdot exp(x^{-\alpha})$ (so for $\alpha=2$ one gets sub-gaussian RVs and for $\alpha=1$ one gets "sub-exponential" RVs). The norm in these spaces controls the tails of the RV, compared to a random variable of density $C\cdot exp(x^{-\alpha})$. Then the norm of a RV $X$ is equivalent (up to universal constants) to the quantity

$$\sup_{p\ge1} {\frac{(\mathbb{E}X^p)^{\frac{1}{p}}}{p^\frac{1}{\alpha}}}$$

I have an overly negative view of this. Higher moments can be useful in some theoretical contexts, but overall they are indeed not used much. This is analogous to the Taylor series of a function. It is rare outside of a few specific areas of math that you ever need anything more than the second order terms in the Taylor series of a function.

• Cubic splines? (although I appreciate your general point) – Yemon Choi Apr 12 '15 at 23:20
• Yemon, you have a point, but I still can't resist pointing out that with cubic splines, you still match up the functions only up to second order at the knot points. Anything more feels like overkill. – Deane Yang Apr 13 '15 at 2:08
• Some of the most widely used ODE solvers are of order 4 (i.e. match 4 derivatives of the solution), while in some applications people use solvers of very high order (up to about 20 or so)... – Martin Hairer Apr 13 '15 at 8:31