Why is free probability a generalization of probability theory?

Question

Note: This question was already asked on Math.SE nearly a week and a half ago but did not receive any responses. To the best of my knowledge, free probability is an active topic of research, so I hope that the level of the question is appropriate for this website. If not, please let me know so I can delete the question myself.

Question: One often sees statements to the effect that "free probability is a generalization of probability theory, which is commutative, to the non-commutative case".

But in what sense does classical probability theory only concern itself with commutative quantities?

If my understanding is correct, and probability theory also deals with non-commutative quantities, then in what sense is free probability a generalization of probability theory?

Context: The simplest random variables are real-valued, and obviously real numbers have commutative multiplication. But random variables can take values in any measurable space (at least this is my understanding and also that of Professor Terry Tao), i.e. random variables can also be random vectors, random matrices, random functions, random measures, random sets, etc. The whole theory of stochastic processes is based on the study of random variables taking values in a space of functions. If the range of the functions of that space is the real numbers, then yes we have commutative multiplication, but I don't see how that's the case if we are e.g. talking about functions into a Riemannian manifold.

EDIT: To clarify what I mean by "classical probability theory", here is Professor Tao's definition of random variable, which is also my understanding of the term (in the most general sense):

Let $R=(R, \mathcal{R})$ be a measurable space (i.e. a set $R$, equipped with a $\sigma$-algebra $\mathcal{R}$ of subsets of $R$). A random variable taking values in $R$ (or an $R$-valued random variable) is a measurable map $X$ from the sample space to $R$, i.e. a function $X: \Omega \to R$ such that $X^{-1}(S)$ is an event for every $S \in \mathcal{R}$.

Then, barring that I am forgetting something obvious, classical probability theory is just the study of random variables (in the above sense).

/EDIT

To be fair though, I don't have a strong understanding of what free probability is. Reading Professor Tao's post about the subject either clarified or confused some things for me, I am not sure which.

In contrast to his other post, where he gives (what seems to me) a more general notion of random variable, in his post about free probability, Professor Tao states that there is a third step to probability theory after assigning a sample space, sigma algebra, and probability measure -- creating a commutative algebra of random variables, which supposedly allows one to define expectations. (1) How does one need a commutative algebra of random variables to define expectations? (2) Since when was defining a commutative algebra of random variables part of Kolmogorov's axiomatization of probability?

Later on his post about free probability, Professor Tao mentions that random scalar variables form a commutative algebra if we restrict to the collection of random variables for which all moments are finite. But doesn't classical probability theory study random variables with non-existent moments? Even in an elementary course I remember learning about the Cauchy distribution.

If so, wouldn't this make classical probability more general than free probability, rather than vice versa, since free probability isn't relevant to, e.g., the Cauchy distribution?

Professor Tao also mentions random matrices (specifically ones with entries which are random scalar variables with all moments finite, if I'm interpreting the tensor product correctly) as an example of a noncommutative algebra which is outside the domain of classical probability but within the scope of free probability. But as I mentioned before, aren't random matrices an object of classical probability theory? As well as random measures, or random sets, or other objects in a measurable space for which there is no notion of multiplication, commutative or non-commutative?

Attempt: Reading Professor Tao's post on free probability further, it seems like the idea might be that certain nice families of random variables can be described by commutative von Neumann algebras. Then free probability generalizes this by studying all von Neumann algebras, including non-commutative ones. The idea that certain nice families of random variables correspond to the (dual category of) commutative von Neumann algebras seems like it is explained in these two answers by Professor Dmitri Pavlov on MathOverflow (1)(2).

But, as Professor Pavlov explains in his answers, commutative von Neumann algebras only correspond to localizable measurable spaces, not arbitrary measurable spaces. While localizable measurable spaces seem like nice objects based on his description, there is one equivalent characterization of them which makes me suspect that they are not the most general objects studied in probability theory: any localizable measurable space "is the coproduct (disjoint union) of points and real lines". This doesn't seem to characterize objects like random functions or random measures or random sets (e.g. point processes), and maybe even not random vectors or matrices, so it does not seem like this is the full scope of classical probability theory.

Thus, if free probability only generalizes the study of localizable measurable spaces, I don't see how it could be considered a generalization of classical probability theory. By considering localizable measurable spaces in the more general framework of possibly non-commutative von Neumann algebras, it might expand the methods employed in probability theory by borrowing tools from functional analysis, but I don't see at all how it expands the scope of the subject. To me it seems like proponents of free probability and quantum probability might be mischaracterizing classical probability and measure theory. More likely I am misinterpreting their statements.

Related question. Professor Pavlov's comments on this article may be relevant.

I am clearly deeply misunderstanding at least one thing, probably several things, here, so any help in identifying where I go wrong would be greatly appreciated.

Answer 1

Quite a lot of questions here!

It is perhaps worth making a distinction between scalar classical probability theory - the study of scalar classical random variables - and more general classical probability theory, in which one studies more general random objects such as random graphs, random sets, random matrices, etc.. The former has the structure of a commutative algebra in addition to an expectation, which allows one to then form many familiar concepts in probability theory such as moments, variances, correlations, characteristic functions, etc., though in many cases one has to impose some integrability condition on the random variables involved in order to ensure that these concepts are well defined; in particular, it can be technically convenient to restrict attention to bounded random variables in order to avoid all integrability issues. In the more general case, one usually does not have the commutative algebra structure, and (in the case of random variables not taking values in a vector space) one also does not have an expectation structure any more.

My focus in my free probability notes is on scalar random variables (commutative or noncommutative), in which one needs both the algebra structure and the expectation structure in order to define the concepts mentioned above. Neither structure is necessary to define the other, but they enjoy some compatibility conditions (e.g. ${\bf E} X^2 \geq 0$ for any real random variable $X$, in both the commutative and noncommutative settings). In my notes, I also restricted largely to the case of bounded random variables $X \in L^\infty$ for simplicity (or at least with random variables $X \in L^{\infty-}$ in which all moments were finite), but one can certainly study unbounded noncommutative random variables as well, though the theory becomes significantly more delicate (much as the spectral theorem becomes significantly more subtle when working with unbounded operators rather than bounded operators).

When teaching classical probability theory, one usually focuses first on the scalar case, and then perhaps moves on to the general case in more advanced portions of the course. Similarly, noncommutative probability (of which free probability is a subfield) usually focuses first on the case of scalar noncommutative variables, which was the also the focus of my post. For instance, random $n \times n$ matrices, using the normalised expected trace $X \mapsto \frac{1}{n} {\bf E} \mathrm{tr} X$ as the trace structure, would be examples of scalar noncommutative random variables (note that the normalised expected trace of a random matrix is a scalar, not a matrix). It is true that random $n \times n$ matrices, when equipped with the classical expectation ${\bf E}$ instead of the normalised expected trace $\frac{1}{n} {\bf E} \mathrm{tr}$, can also be viewed as classical non-scalar random variables, but this is a rather different structure (note now that the expectation is a matrix rather than a scalar) and should not be confused with the scalar noncommutative probability structure one can place here.

It is certainly possible to consider non-scalar noncommutative random variables, such as a matrix in which the entries are themselves elements of some noncommutative tracial von Neumann algebra (e.g. a matrix of random matrices); see e.g. Section 5 of these slides of Speicher. Similarly, there is certainly literature on free point processes (see e.g. this paper), noncommutative white noise (see e.g. this paper), etc., but these are rather advanced topics and beyond the scope of the scalar noncommutative probability theory discussed in my notes. I would not recommend trying to think about these objects until one is completely comfortable conceptually both with non-scalar classical random variables and with scalar noncommutative random variables, as one is likely to become rather confused otherwise when dealing with them. (This is analogous to how one should not attempt to study quantum field theory until one is completely comfortable conceptually both with classical field theory and with the quantum theory of particles. Much as one should not conflate the superficially similar notions of a classical field and a quantum wave function, one should also not conflate the superficially similar notions of a non-scalar classical random variable and a scalar noncommutative random variable.)

Regarding localisable measurable spaces: all standard probability spaces generate localisable measurable spaces. Technically, it is true that there do exist some pathological probability spaces whose corresponding measurable spaces are not localisable; however the vast majority of probability theory can be conducted on standard probability spaces, and there are some technical advantages to doing so, particularly when it comes to studying conditional expectations with respect to continuous random variables or continuous $\sigma$-algebras.

Answer 2

There have been many good answers to this question, but it might be that the main point gets lost in too many details. So, as kind of expert on free probability theory, let me try to give a short direct answer to the question “Why is free probability a generalization of probability theory.”

There are two main ingredients in free probability theory: first, the general notion of a non-commutative probability space and second, the more specific notion of freeness (or free independence).

A non-commutative probability space consists of an algebra and a linear functional. The algebra can (despite the use of “non-commutative”) also be commutative and thus a classical probability space (encoded in the commutative algebra of random variables and the functional given by taking the expectation with respect to the underlying probability measure) is also an example of a non-commutative probability space. However, in this generality non-commutative (as well as classical) probability spaces are not too exciting. One needs more structure for interesting statements. In the classical setting, the most basic additional structure is “independence”. In free probability the corresponding structure is “free independence”. However, free independenc is NOT a generalization of independence; it is an analogue. What independence means for classical (commuting) random variables, free independence means for non-commuting variables. Apart from trivial situations, there are no classical random variables which are free. Hence freeness is not a kind of dependence for classical variables; it is a special relation for non-commuting variables, which behaves in many respects like independence.

Hence the above question has two possible answers, depending on how it is interpreted.

Read as “Why is a non-commutative probability space a generalization of a classical probability space?” the answer is just: because a commutative algebra is also allowed as an example of a non-commutative algebra.

Read as “Why is free independence a generalization of classical independence?” the answer is: this is actually not true, free independence is not a generalization, but an analogue of classical independence.

Answer 3

First of all, you are mixing many questions into one post...

It depends on what do you mean by "generalization". And I am not sure what you mean by talking about "commutative" without mentioning notion of "exchageability" or "conditional independence".

The classical probability theory dealt with dependent random variables, but usually they are discussed in stochastic process like autocorrelation process, where the dependent relation is tractable. In free probability, the dependence could be wilder.

But in what sense does classical probability theory only concern itself with commutative quantities?

In short it discussed mostly nothing beyond exchageability.

Free probability is one of many possible generalizations of the notion of exchangeability. There are many other generalization of the notion of exchangeability, for example the exchageable pairs. Free probability provides a method that deals with such a "non-commutative relations". But free probability is yet not the only method that deals with dependence more than exchangeablility.

From the perspective of a statistician, free probability is a natural generalization of the notion of exchangeability. Generalization of de Finetti's Theorem is one very interesting application of free probability framework. If you are a Bayesian, a natural question to ask is how to justify the conditional independence assumption in model building. de Finetti's theorem is a strong justification that why putting a prior on the exchangeable sequence is natural. After this justification, some people asked what if we have a weaker assumption than exchangeability? Then we asked what can we assert if a pair of random variable is "non-commutative" in some sense.

From perspective of a probabilist, I understand why you interpret free probability in that way. You can regard free probability as a general framework that includes, say, matrix valued random variables. In that way you can also treat $W^*$-algebra of bounded random operators on the sample space $X$ (Hilbert space) with specified value space $Y$(say matrix space) as a generalization of $M(X)$ the space of collection of probability measures on $X$. Then free probability is a formalism of the notion of (conditional independence) There is a monograph talked about this view in depth.

But doesn't classical probability theory study random variables with non-existent moments? Even in an elementary course I remember learning about the Cauchy distribution.

The reason why Tao's post is limited to bounded case is partially due to the nice correspondence of $W^*$-algebra and the formalism that I mentioned above. Your claim seems a bit odd to me. $L^p$ spaces do not include many real valued functions; Sobolev spaces do not include many $L^p$ functions, are they generalizations of the former notions?