# Why is free probability a generalization of probability theory?

Note: This question was already asked on Math.SE nearly a week and a half ago but did not receive any responses. To the best of my knowledge, free probability is an active topic of research, so I hope that the level of the question is appropriate for this website. If not, please let me know so I can delete the question myself.

Question: One often sees statements to the effect that "free probability is a generalization of probability theory, which is commutative, to the non-commutative case".

But in what sense does classical probability theory only concern itself with commutative quantities?

If my understanding is correct, and probability theory also deals with non-commutative quantities, then in what sense is free probability a generalization of probability theory?

Context: The simplest random variables are real-valued, and obviously real numbers have commutative multiplication. But random variables can take values in any measurable space (at least this is my understanding and also that of Professor Terry Tao), i.e. random variables can also be random vectors, random matrices, random functions, random measures, random sets, etc. The whole theory of stochastic processes is based on the study of random variables taking values in a space of functions. If the range of the functions of that space is the real numbers, then yes we have commutative multiplication, but I don't see how that's the case if we are e.g. talking about functions into a Riemannian manifold.

EDIT: To clarify what I mean by "classical probability theory", here is Professor Tao's definition of random variable, which is also my understanding of the term (in the most general sense):

Let $R=(R, \mathcal{R})$ be a measurable space (i.e. a set $R$, equipped with a $\sigma$-algebra $\mathcal{R}$ of subsets of $R$). A random variable taking values in $R$ (or an $R$-valued random variable) is a measurable map $X$ from the sample space to $R$, i.e. a function $X: \Omega \to R$ such that $X^{-1}(S)$ is an event for every $S \in \mathcal{R}$.

Then, barring that I am forgetting something obvious, classical probability theory is just the study of random variables (in the above sense).

/EDIT

To be fair though, I don't have a strong understanding of what free probability is. Reading Professor Tao's post about the subject either clarified or confused some things for me, I am not sure which.

In contrast to his other post, where he gives (what seems to me) a more general notion of random variable, in his post about free probability, Professor Tao states that there is a third step to probability theory after assigning a sample space, sigma algebra, and probability measure -- creating a commutative algebra of random variables, which supposedly allows one to define expectations. (1) How does one need a commutative algebra of random variables to define expectations? (2) Since when was defining a commutative algebra of random variables part of Kolmogorov's axiomatization of probability?

Later on his post about free probability, Professor Tao mentions that random scalar variables form a commutative algebra if we restrict to the collection of random variables for which all moments are finite. But doesn't classical probability theory study random variables with non-existent moments? Even in an elementary course I remember learning about the Cauchy distribution.

If so, wouldn't this make classical probability more general than free probability, rather than vice versa, since free probability isn't relevant to, e.g., the Cauchy distribution?

Professor Tao also mentions random matrices (specifically ones with entries which are random scalar variables with all moments finite, if I'm interpreting the tensor product correctly) as an example of a noncommutative algebra which is outside the domain of classical probability but within the scope of free probability. But as I mentioned before, aren't random matrices an object of classical probability theory? As well as random measures, or random sets, or other objects in a measurable space for which there is no notion of multiplication, commutative or non-commutative?

Attempt: Reading Professor Tao's post on free probability further, it seems like the idea might be that certain nice families of random variables can be described by commutative von Neumann algebras. Then free probability generalizes this by studying all von Neumann algebras, including non-commutative ones. The idea that certain nice families of random variables correspond to the (dual category of) commutative von Neumann algebras seems like it is explained in these two answers by Professor Dmitri Pavlov on MathOverflow (1)(2).

But, as Professor Pavlov explains in his answers, commutative von Neumann algebras only correspond to localizable measurable spaces, not arbitrary measurable spaces. While localizable measurable spaces seem like nice objects based on his description, there is one equivalent characterization of them which makes me suspect that they are not the most general objects studied in probability theory: any localizable measurable space "is the coproduct (disjoint union) of points and real lines". This doesn't seem to characterize objects like random functions or random measures or random sets (e.g. point processes), and maybe even not random vectors or matrices, so it does not seem like this is the full scope of classical probability theory.

Thus, if free probability only generalizes the study of localizable measurable spaces, I don't see how it could be considered a generalization of classical probability theory. By considering localizable measurable spaces in the more general framework of possibly non-commutative von Neumann algebras, it might expand the methods employed in probability theory by borrowing tools from functional analysis, but I don't see at all how it expands the scope of the subject. To me it seems like proponents of free probability and quantum probability might be mischaracterizing classical probability and measure theory. More likely I am misinterpreting their statements.

Related question. Professor Pavlov's comments on this article may be relevant.

I am clearly deeply misunderstanding at least one thing, probably several things, here, so any help in identifying where I go wrong would be greatly appreciated.

• I would think that you need talk about "independence" to understand what free probability theory is about: if two statistically independent matrices $A$ and $B$ commute, then the probability distribution of the eigenvalues of $A+B$ follows from classical probability theory as the convolution of the individual distributions; free probability theory generalizes this to the case that $A$ and $B$ do not commute. – Carlo Beenakker Apr 22 '17 at 18:00
• @CarloBeenakker A very good example indeed! – Henry.L Apr 22 '17 at 18:53
• I think "classical probability" is meant to indicate real-valued random variables. – Qiaochu Yuan Apr 22 '17 at 18:58
• @QiaochuYuan The OP obviously refers "classic" more than that since he classified stochastic processes as "classic" by saying "...This doesn't seem to characterize objects like random functions or random measures or random sets (e.g. point processes), and maybe even not random vectors or matrices, so it does not seem like this is the full scope of classical probability theory..." I am not quite sure what he meant by "classic" so I wrote down a few sharp contrast points. – Henry.L Apr 22 '17 at 19:15
• @CarloBeenakker I think I agree, although I am not entirely certain I understand your point. Independence can be (is best) defined in terms of sigma algebras, and then two random variables (including in the wider sense of random sets, random measures, etc.) are independent if and only if the sigma algebras they generate are independent. So does free probability only generalize that part of probability theory for which the random variables take values in some ring and thus independence implies some convolution formula for the distribution of their sum? – Chill2Macht Apr 23 '17 at 8:15

Quite a lot of questions here!

It is perhaps worth making a distinction between scalar classical probability theory - the study of scalar classical random variables - and more general classical probability theory, in which one studies more general random objects such as random graphs, random sets, random matrices, etc.. The former has the structure of a commutative algebra in addition to an expectation, which allows one to then form many familiar concepts in probability theory such as moments, variances, correlations, characteristic functions, etc., though in many cases one has to impose some integrability condition on the random variables involved in order to ensure that these concepts are well defined; in particular, it can be technically convenient to restrict attention to bounded random variables in order to avoid all integrability issues. In the more general case, one usually does not have the commutative algebra structure, and (in the case of random variables not taking values in a vector space) one also does not have an expectation structure any more.

My focus in my free probability notes is on scalar random variables (commutative or noncommutative), in which one needs both the algebra structure and the expectation structure in order to define the concepts mentioned above. Neither structure is necessary to define the other, but they enjoy some compatibility conditions (e.g. ${\bf E} X^2 \geq 0$ for any real random variable $X$, in both the commutative and noncommutative settings). In my notes, I also restricted largely to the case of bounded random variables $X \in L^\infty$ for simplicity (or at least with random variables $X \in L^{\infty-}$ in which all moments were finite), but one can certainly study unbounded noncommutative random variables as well, though the theory becomes significantly more delicate (much as the spectral theorem becomes significantly more subtle when working with unbounded operators rather than bounded operators).

When teaching classical probability theory, one usually focuses first on the scalar case, and then perhaps moves on to the general case in more advanced portions of the course. Similarly, noncommutative probability (of which free probability is a subfield) usually focuses first on the case of scalar noncommutative variables, which was the also the focus of my post. For instance, random $n \times n$ matrices, using the normalised expected trace $X \mapsto \frac{1}{n} {\bf E} \mathrm{tr} X$ as the trace structure, would be examples of scalar noncommutative random variables (note that the normalised expected trace of a random matrix is a scalar, not a matrix). It is true that random $n \times n$ matrices, when equipped with the classical expectation ${\bf E}$ instead of the normalised expected trace $\frac{1}{n} {\bf E} \mathrm{tr}$, can also be viewed as classical non-scalar random variables, but this is a rather different structure (note now that the expectation is a matrix rather than a scalar) and should not be confused with the scalar noncommutative probability structure one can place here.

It is certainly possible to consider non-scalar noncommutative random variables, such as a matrix in which the entries are themselves elements of some noncommutative tracial von Neumann algebra (e.g. a matrix of random matrices); see e.g. Section 5 of these slides of Speicher. Similarly, there is certainly literature on free point processes (see e.g. this paper), noncommutative white noise (see e.g. this paper), etc., but these are rather advanced topics and beyond the scope of the scalar noncommutative probability theory discussed in my notes. I would not recommend trying to think about these objects until one is completely comfortable conceptually both with non-scalar classical random variables and with scalar noncommutative random variables, as one is likely to become rather confused otherwise when dealing with them. (This is analogous to how one should not attempt to study quantum field theory until one is completely comfortable conceptually both with classical field theory and with the quantum theory of particles. Much as one should not conflate the superficially similar notions of a classical field and a quantum wave function, one should also not conflate the superficially similar notions of a non-scalar classical random variable and a scalar noncommutative random variable.)

Regarding localisable measurable spaces: all standard probability spaces generate localisable measurable spaces. Technically, it is true that there do exist some pathological probability spaces whose corresponding measurable spaces are not localisable; however the vast majority of probability theory can be conducted on standard probability spaces, and there are some technical advantages to doing so, particularly when it comes to studying conditional expectations with respect to continuous random variables or continuous $\sigma$-algebras.

• Thanks for a very clarifying answer! Two questions, If we are discussing within a std prob space, what is the difference between your "noncommutative rv" and "inexchangeable rv"? And is it really necessary to have an integral representation (expectation) to discuss commutativity within free prob? – Henry.L Apr 24 '17 at 17:19
• (a) Standard probability spaces only appear in classical probability theory; noncommutative probability theory does not have any underlying probability space (though, through tools such as the GNS construction, one can create a Hilbert space that is somewhat analogous to the classical probability space, in much the same way that the Hilbert space of quantum states is analogous to classical phase space). ... – Terry Tao Apr 24 '17 at 17:57
• (b) Classical exchangeability is related to classical independence via de Finetti's theorem, and there is a notion of free exchangeability that is similarly related to free independence via a free version of de Finetti's theorem (see e.g. arxiv.org/abs/0807.0677). So I would view the concept of exchangeability (both classical and free) as being orthogonal to the classical probability / free probability distinction; also, as pointed out in the previous post, there are also many other ways to modify the notion of exchangeability. – Terry Tao Apr 24 '17 at 17:58
• (c) One could consider noncommutative probability in which traces are not necessarily defined, much as one can consider the commutativity (or lack thereof) of operators on a Hilbert space that are not necessarily of trace class. For instance, one could define a notion of freeness of variables $X,Y$ that are not of finite trace, but nevertheless have a functional calculus, by declaring $f(X), g(Y)$ to be free for any bounded $f,g$ in this calculus. But one has to take care as some free probability theorems may now fail in this unbounded setting. – Terry Tao Apr 24 '17 at 18:01
• @Henry.L Consider for instance the random $2 \times 2$ matrix $X := \begin{pmatrix} x & 0 \\ 0 & -x \end{pmatrix}$, where $x$ is drawn uniformly from $[0,1]$. Using the classical expectation structure ${\bf E}$, $X$ has non-zero mean $\begin{pmatrix} 1/2 & 0 \\ 0 & -1/2 \end{pmatrix}$. But using the noncommutative trace structure $\frac{1}{2} {\bf E} \mathrm{tr}$, $X$ has mean zero (or trace zero, if one prefers). – Terry Tao Apr 25 '17 at 16:27

There have been many good answers to this question, but it might be that the main point gets lost in too many details. So, as kind of expert on free probability theory, let me try to give a short direct answer to the question “Why is free probability a generalization of probability theory.”

There are two main ingredients in free probability theory: first, the general notion of a non-commutative probability space and second, the more specific notion of freeness (or free independence).

A non-commutative probability space consists of an algebra and a linear functional. The algebra can (despite the use of “non-commutative”) also be commutative and thus a classical probability space (encoded in the commutative algebra of random variables and the functional given by taking the expectation with respect to the underlying probability measure) is also an example of a non-commutative probability space. However, in this generality non-commutative (as well as classical) probability spaces are not too exciting. One needs more structure for interesting statements. In the classical setting, the most basic additional structure is “independence”. In free probability the corresponding structure is “free independence”. However, free independenc is NOT a generalization of independence; it is an analogue. What independence means for classical (commuting) random variables, free independence means for non-commuting variables. Apart from trivial situations, there are no classical random variables which are free. Hence freeness is not a kind of dependence for classical variables; it is a special relation for non-commuting variables, which behaves in many respects like independence.

Hence the above question has two possible answers, depending on how it is interpreted.

Read as “Why is a non-commutative probability space a generalization of a classical probability space?” the answer is just: because a commutative algebra is also allowed as an example of a non-commutative algebra.

Read as “Why is free independence a generalization of classical independence?” the answer is: this is actually not true, free independence is not a generalization, but an analogue of classical independence.

First of all, you are mixing many questions into one post...

It depends on what do you mean by "generalization". And I am not sure what you mean by talking about "commutative" without mentioning notion of "exchageability" or "conditional independence".

The classical probability theory dealt with dependent random variables, but usually they are discussed in stochastic process like autocorrelation process, where the dependent relation is tractable. In free probability, the dependence could be wilder.

But in what sense does classical probability theory only concern itself with commutative quantities?

In short it discussed mostly nothing beyond exchageability.

Free probability is one of many possible generalizations of the notion of exchangeability. There are many other generalization of the notion of exchangeability, for example the exchageable pairs. Free probability provides a method that deals with such a "non-commutative relations". But free probability is yet not the only method that deals with dependence more than exchangeablility.

From the perspective of a statistician, free probability is a natural generalization of the notion of exchangeability. Generalization of de Finetti's Theorem is one very interesting application of free probability framework. If you are a Bayesian, a natural question to ask is how to justify the conditional independence assumption in model building. de Finetti's theorem is a strong justification that why putting a prior on the exchangeable sequence is natural. After this justification, some people asked what if we have a weaker assumption than exchangeability? Then we asked what can we assert if a pair of random variable is "non-commutative" in some sense.

From perspective of a probabilist, I understand why you interpret free probability in that way. You can regard free probability as a general framework that includes, say, matrix valued random variables. In that way you can also treat $W^*$-algebra of bounded random operators on the sample space $X$ (Hilbert space) with specified value space $Y$(say matrix space) as a generalization of $M(X)$ the space of collection of probability measures on $X$. Then free probability is a formalism of the notion of (conditional independence) There is a monograph talked about this view in depth.

But doesn't classical probability theory study random variables with non-existent moments? Even in an elementary course I remember learning about the Cauchy distribution.

The reason why Tao's post is limited to bounded case is partially due to the nice correspondence of $W^*$-algebra and the formalism that I mentioned above. Your claim seems a bit odd to me. $L^p$ spaces do not include many real valued functions; Sobolev spaces do not include many $L^p$ functions, are they generalizations of the former notions?

• With regards to your last question, it would seem that they are not. At best, they are "orthogonal", i.e. a generalization of a special case, but not a generalization of the entire class. Maybe a better analogy would be made with the relationship with algebraic varieties and smooth manifolds. I guess my question is whether I have been learning probability theory "incorrectly" up to now, since in standard graduate texts on the subject free probability theory isn't mentioned. Also the paper you linked to looks very interesting, Professor Jordan is one of my favorite professors. – Chill2Macht Apr 23 '17 at 8:24
• I guess I'm still confused though since I don't see how the study of random matrices is general enough to include as a special case random measures or point processes. I remember learning briefly about random matrices in my graduate probability course, although free probability wasn't mentioned, but my professor didn't seem to state that random matrices generalized everything else in probability theory. Also I remember covering exchangeable random variables briefly, or rather just the 0-1 law for exchangeable sigma algebras, and I have read about de Finetti's theorems once or twice, but don't – Chill2Macht Apr 23 '17 at 8:27
• quite grasp their significance yet. At least, from the exchangeable 0-1 law, I got the impression that there were a lot of objects studied in probability theory for which there was "more to the story" than the exchangeable sigma algebra, but that was more than a year ago, so my memory is somewhat hazy at this point, since I haven't yet had the opportunity to apply any of that knowledge further. Also I apologize about asking so many questions -- my intent was just to have people only focus on the questions in gray, and the remaining questions were there for "flavor". – Chill2Macht Apr 23 '17 at 8:29
• In any case though, thank you for pointing out what is actually meant by "commutative", i.e. that it is referring to exchangeable random variables (and conditional dependence somehow) -- I had previously thought that it was referring to random variables taking values in a commutative ring. – Chill2Macht Apr 23 '17 at 8:43
• @Chill2Macht (1)My point is that such "generalization" allows finer structure but not exactly an inclusion. (2)Free probability is still research front so it is not expected to include in grad prob (3)random matrices does not generalized anything(and I did not claimed that) (4)Okay, definitely not taking values in commutative ring, but there is a deeper representation theoretic characterization; we should probably discussed via emails if you want. (leave it below) :) – Henry.L Apr 23 '17 at 11:27