Perhaps a related question is Why are matrices ubiquitous but hypermatrices rare? in the sense that higher-order moments can be regarded as hypermatrices or tensors. For example, the second order moment of a random variable $x\in\mathbb{R}^n$ might be written $\mathbb{E}[x\otimes x]$. The third order moment is $\mathbb{E}[x\otimes x\otimes x]$, and so on.
The (relatively) lack of extensive use of higher order moments may be because questions about them inherit the difficulties that come with all tensor problems (Most tensor problems are NP-hard).
A very clear example of higher order moments in use is independent component analysis (ICA), which essentially amounts to a tensor decomposition of the fourth order cumulant (see here, for example). ICA is an instance of a slightly more general problem known as blind source separation, which has wide-ranging applications.