# Is there an introduction to probability theory from a structuralist/categorical perspective?

The title really is the question, but allow me to explain.

I am a pure mathematician working outside of probability theory, but the concepts and techniques of probability theory (in the sense of Kolmogorov, i.e., probability measures) are appealing and potentially useful to me. It seems to me that, perhaps more than most other areas of mathematics, there are many, many nice introductory (as well as not so introductory) texts on this subject.

However, I haven't found any that are written from what it is arguably the dominant school of thought of contemporary mainstream mathematics, i.e., from a structuralist (think Bourbaki) sensibility. E.g., when I started writing notes on the texts I was reading, I soon found that I was asking questions and setting things up in a somewhat different way. Here are some basic questions I couldn't stop from asking myself:

[0) Define a Borel space to be a set $X$ equipped with a $\sigma$-algebra of subsets of $X$. This is already not universally done (explicitly) in standard texts, but from a structuralist approach one should gain some understanding of such spaces before one considers the richer structure of a probability space.]

1) What is the category of Borel spaces, i.e., what are the morphisms? Does it have products, coproducts, initial/final objects, etc? As a significant example here I found the notion of the product Borel space -- which is exactly what you think if you know about the product topology -- but seemed underemphasized in the standard treatments.

2) What is the category of probability spaces, or is this not a fruitful concept (and why?)? For instance, a subspace of a probability space is, apparently, not a probability space: is that a problem? Is the right notion of morphism of probability spaces a measure-preserving function?

3) What are the functorial properties of probability measures? E.g., what are basic results on pushing them forward, pulling them back, passing to products and quotients, etc. Here again I will mention that product of an arbitrary family of probability spaces -- which is a very useful-looking concept! -- seems not to be treated in most texts. Not that it's hard to do: see e.g.

http://www.math.uga.edu/~pete/saeki.pdf

I am not a category theorist, and my taste for how much categorical language to use is probably towards the middle of the spectrum: that is, I like to use a very small categorical vocabulary (morphisms, functors, products, coproducts, etc.) as often as seems relevant (which is very often!). It would be a somewhat different question to develop a truly categorical take on probability theory. There is definitely some nice mathematics here, e.g. I recall an arxiv article (unfortunately I cannot put my hands on it at this moment) which discussed independence of events in terms of tensor categories in a very persuasive way. So answers which are more explicitly categorical are also welcome, although I wish to be clear that I'm not asking for a categorification of probability theory per se (at least, not so far as I am aware!).

-
I am certainly not an expert, but I was looking for a similar thing, and found Dudley's book (books.google.com/…) promising. He doesn't mention categories at all, but it seems that he has them in mind. In particular, he defined "measurable function" between any two measurable spaces (p. 116), [which is different from the definition in Rudin]. Also, while he proves the existence of countable products of probability spaces, he does remark on converting the proof to an arbitrary product (p. 259). –  user2734 Apr 8 '10 at 16:16
This is not developed enough to be a (partial) answer rather than a comment, but see perhaps: golem.ph.utexas.edu/category/2007/02/… (and other google/Mathscinet results for "Giry monad") –  Yemon Choi Apr 8 '10 at 16:16
One thing I thought I'd mention - as a probablist manqué - is a comment at the beginning of Williams' Probability with Martingales, where he says something along lines of "it would be nice if we could think of random variables as equivalence classes of functions rather than functions, so that we don't need to keep inserting 'a.e.' everywhere; but this point of view runs into trouble when dealing with continuous-time stochastic processes". Which implies he is not keen on 'structuralist POV', although it doesn't rule out the possibility. –  Yemon Choi Apr 8 '10 at 16:20
Something that you may want to consider is the fact that probability spaces are not the essential objects in probability, for at least two reasons. First, it is very common to change the underlying probability space, as long as the distributions of the relevant random variables remain the same. This allows to consider new events along the way. As suggested in Neel's answer, this may have a categorical formulation. But worst than that is the fact that often (every time martingales appear, at least) you want to leave the space unchanged and vary the sigma-algebra. –  Andrea Ferretti Apr 9 '10 at 16:29
Indeed, one of the major differences between measure theory and probability theory (besides the perspective being completely different) is that in measure theory one fixes one sigma algebra, and in probability one considers relationships between multiple sigma algebras. –  Mark Meckes Apr 9 '10 at 16:54

One can argue that an object of the right category of spaces in measure theory is not a set equipped with a $\sigma$-algebra of measurable sets, but rather a set $S$ equipped with a $\sigma$-algebra $M$ of measurable sets and a $\sigma$-ideal $N$ of $M$ consisting of sets of measure $0$. The reason for this is that you can hardly state any theorem of measure theory or probability theory without referring to sets of measure $0$. However, objects of this category contain less data than the usual measured spaces, because they are not equipped with a measure. Therefore I prefer to call them measurable spaces. A morphism of measurable spaces $(S,M,N)\to(T,P,Q)$ is a map $S\to T$ such that the preimage of every element of $P$ is a union of an element of $M$ and a subset of an element of $N$ and the preimage of every element of $Q$ is a subset of an element of $N$.

Irving Segal proved that for a measurable space the following properties are equivalent:

1. The Boolean algebra $M/N$ of equivalence classes of measurable sets is complete;
2. The space of equivalence classes of all bounded (or unbounded) real-valued functions on $S$ is Dedekind-complete;
3. Radon-Nikodym theorem is true for $(S,M,N)$;
4. Riesz theorem is true for $(S,M,N)$;
5. Equivalence classes of bounded functions on $S$ form a von Neumann algebra (aka $W^*$-algebra).
6. $(S,M,N)$ is a coproduct (disjoint union) of points and real lines.

A measurable space that satisfies these conditions is called localizable.

This theorem tells us that if we want to prove anything nontrivial about measurable spaces, we better restrict ourselves to localizable measurable spaces. We also have a nice illustration of the claim I made in the first paragraph: None of these statements would be true without identifying objects that differ on a set of measure $0$. For example, take a non-measurable set $G$ and a family of one-element subsets of $G$ indexed by themselves. This family of measurable sets does not have a supremum in the Boolean algebra of measurable sets, thus disproving a naïve version of (1).

Another argument for restricting to localizable spaces is the following version of Gelfand-Neumark theorem:

The category of localizable measurable spaces is equivalent to the category of commutative von Neumann algebras (aka $W^*$-algebras) and their morphisms (normal unital homomorphisms of $^*$-algebras).

I actually prefer to define the category of localizable measurable spaces as the opposite category of the category of commutative $W^*$-algebras. The reason for this is that the classical definition of measurable space exhibits immediate connections only to descriptive set theory (and with additional effort to Boolean algebras), which are mostly irrelevant for the central core of mathematics, whereas the description in terms of operator algebras immediately connects measure theory to other areas of the central core (noncommutative geometry, algebraic geometry, complex geometry, differential geometry etc.). Also it is easier to use in practice. Let me illustrate this statement with just one example: When we try to define measurable bundles of Hilbert spaces on a localizable measurable space set-theoretically, we run into all sorts of problems if the fibers can be non-separable, and I do not know how to fix this problem in the set-theoretic framework. On the other hand, in the algebraic framework we can simply say that a bundle of Hilbert spaces is a Hilbert module over the corresponding $W^*$-algebra.

Categorical properties of $W^*$-algebras (hence of localizable measurable spaces) were investigated by Guichardet. Electronic version of this paper is available here. Let me mention some of his results. The category of localizable measurable spaces admits equalizers and coequalizers, arbitrary coproducts and hence arbitrary colimits. It also admits products, although they are quite different from what one might think. For example, the product of two real lines is not $\Bbb R^2$ with the two obvious projections. The product contains $\Bbb R^2$, but it also has a lot of other stuff, for example, the diagonal of $\Bbb R^2$, which is needed to satisfy the universal property for the two identity maps on $\Bbb R$. The more intuitive product of measurable spaces ($\Bbb R\times \Bbb R=\Bbb R^2$) corresponds to the spatial tensor product of von Neumann algebras and forms a part of a symmetric monoidal structure on the category of measurable spaces. See Guichardet's paper for other categorical properties (monoidal structures on measurable spaces, flatness, existence of filtered limits, etc.).

Finally let me mention pushforward and pullback properties of measures on measurable spaces. I will talk about more general case of $L^p$ spaces instead of just measures (i.e., $L^1$ spaces). For the sake of convenience let $L_p(M) := L^{1/p}(M)$, where $M$ is a measurable space. Here $p$ can be an arbitrary complex number with a nonnegative real part. Note that you don't need a measure on $M$ to define $L_p(M)$. In particular, $L_0$ is the space of all bounded functions (i.e., the $W^*$-algebra itself), $L_1$ is the space of finite complex-valued measures (the dual of $L_0$ in the $\sigma$-weak topology), and $L_{1/2}$ is the Hilbert space of half-densities. I will also talk about extended positive part $E^+L_p$ of $L_p$ for real $p$. In particular, $E^+L_1$ is the space of all (not necessarily finite) positive measures.

Pushforward for $L_p$ spaces: Suppose we have a morphism of measurable spaces $M\to N$. If $p=1$, then we have a canonical map $L_1(M) \to L_1(N)$, which just the dual of $L_0(N) \to L_(M)$ in the $\sigma$-weak topology. Geometrically, this is the fiberwise integration map. If $p\neq 1$, then we only have a pushforward map of the extended positive parts: $E^+L_p(M) \to E^+L_p(N)$, which is non-additive unless $p=1$. Geometrically, this is the fiberwise $L_p$ norm. Thus $L_1$ is a functor from the category of measurable spaces to the category of Banach spaces and $E^+L_p$ is a functor to the category of "positive homogeneous $p$-cones". The pushforward map preserves the trace on $L_1$ and hence sends a probability measure to a probability measure.

To define pullback of $L_p$ spaces (in particular, $L_1$ spaces) one needs to pass to a different category of measurable spaces. In algebraic language, if we have two $W^*$-algebras $A$ and $B$, then a morphism from $A$ to $B$ is a usual morphism of $W^*$-algebras $f: A\to B$ together with an operator valued weight $T: E^+(B)→E^+(A)$ associated to $f$. Here $E^+(A)$ denotes the extended positive part of A (think of positive functions on $\mathrm{Spec} A$ that can take infinite values). Geometrically, this is a morphism $\mathrm{Spec} f: \mathrm{Spec} B \to \mathrm{Spec} A$ between the corresponding measurable spaces and a choice of measure on each fiber of $\mathrm{Spec} f$. Now we have a canonical additive map $E^+L_p(\mathrm{Spec} A) \to E^+L_p(\mathrm{Spec} B)$, which makes $E^+L_p$ into a contravariant functor from the category of measurable spaces equipped with a fiberwise measure to the category of “positive homogeneous additive cones”.

If we want to have a pullback of $L_p$ spaces themselves and not just their extended positive parts, we need to replace operator valued weights in the above definition by finite complex-valued operator valued weights $T: B \to A$ (think of fiberwise complex-valued measure). Then $L_p$ becomes a functor from the category of measurable spaces to the category of Banach spaces (if the real part of $p$ is at most $1$) or quasi-Banach spaces (if the real part of $p$ is greater than $1$). Here $p$ is an arbitrary complex number with a nonnegative real part. Notice that for $p=0$ we get the original map $f: A\to B$ and in this (and only this) case we don't need $T$.

Finally, if we restrict ourselves to an even smaller subcategory of measurable spaces equipped with a finite operator valued weight T such that $T(1)=1$ (i.e., T is a conditional expectation; think of fiberwise probability measure), then the pullback map preserves the trace on $L_1$ and in this case the pullback of a probability measure is a probability measure.

There is also a smooth analog of the theory described above: The category of measurable spaces and their morphisms is replaced by the category of smooth manifolds and submersions, $L_p$ spaces are replaced by bundles of $p$-densities, operator valued weights are replaced by sections of the bundle of relative $1$-densities, integration map on $1$-densities is defined via Poincaré duality (to avoid circular dependence on measure theory) etc. The forgetful functor that sends a smooth manifold to its underlying measurable space commutes with everything and preserves everything.

-
This is great! I wish I could vote you up multiple times. –  Neel Krishnaswami Apr 9 '10 at 8:44
I agree. Great answer! –  George Lowther Apr 9 '10 at 21:34

In the spirit of this answer to a different question, I'll offer a contrarian answer. How to understand probability theory from a structuralist perspective:

Don't.

To put it less provocatively, what I really mean is that probabilists don't think about probability theory that way, which is why they don't write their introductory books that way. The reason probabilists don't think that way is that probability theory is not about probability spaces. Probability theory is about families of random variables. Probability spaces are the mathematical formalism used to talk about random variables, but most probabilists keep the probability spaces in the background as much as possible. Doing probability theory while dwelling on probability spaces is a little like doing number theory while dwelling on a definition of 1 as $\{\{\}\}$ etc. (That last sentence is definitely an overstatement, but I can't think of a more apt analogy offhand.)

That said, multiple perspectives are always good to have, so I'm very happy you asked this question and that you've gotten some very nice noncontrarian answers that I hope to digest better myself.

Added: Here is something which is perhaps more similar to dwelling on probability spaces. To set the stage for graph theory carefully one may start by defining a graph as a pair $(V,E)$ in which $V$ is a (finite, nonempty) set and $E$ is a set of cardinality 2 subsets of $V$. You need to start tweaking this in various ways to allow loops, directed graphs, multigraphs, infinite graphs, etc. But worrying about the details of how you do this is a distraction from actually doing graph theory.

-
Indeed, I saw a quote from somebody famous (if I think of the author I'll edit) to the effect that "one could say that probability theory is the study of measure spaces with measure one, but this is like saying that number theory is the study of finite strings of the digits {0,...,9}." –  Nate Eldredge May 25 '10 at 13:36
Another great quote along the same lines, from Rudin (Real and Complex Analysis, page 18 in my edition): "For instance, the real line may be described as a quadruple $(R^1, +, \cdot, <)$ where $+$, $\cdot$, and $<$ satisfy the axioms of a complete archimedean ordered field. But it is a safe bet that very few mathematicians think of the real field as an ordered quadruple." –  Carl Offner May 30 '10 at 14:47
That is a great quote, but it doesn't make all the points Nate's quote does. If you think of the reals as a quadruple, you have the formalism necessary to understand and prove theorems about real numbers, although you may lack the intuition needed to appreciate the theorems. But if you think of natural numbers as strings of digits, you're missing not only intuition but also interesting algebraic structure. Likewise, a measure space with measure one is insufficient structure for probability; you need some additional algebraic or geometric structure before you can even talk about expectations. –  Mark Meckes May 30 '10 at 23:35
You can't talk about expectations if all you have is a probability space. You need to look at a measurable function (random variable) from your probability space into $\mathbb{R}$ or a similar algebraic structure; or equivalently you need your probability space itself to have some algebraic structure. –  Mark Meckes May 31 '10 at 3:16
Just for the record, that quote is my own, though the general sentiment that probability is not about measure spaces is certainly very widely held among probabilists. –  Terry Tao Sep 3 '10 at 19:08

A few months ago, Terry Tao had a really insightful post about "the probabilistic way of thinking", in which he suggested that a nice category of probability spaces was one in which the objects were probability spaces and the morphisms were extensions (ie, measurable surjections which are probability preserving). By avoiding looking at the details of the sample space, you can elegantly capture the style of probabilistic arguments in which you introduce new sources of randomness as needed.

-
This is a very interesting (and impressively long!) set of notes; thanks for linking to them. I may make more comments after I've digested them. –  Pete L. Clark Apr 9 '10 at 3:11

As already noted, most probabilists identify random variables essentially with their distribution. The problem is that the kind of operations one can do with random variables often depend on the spaces they are defined on. The probabilitys spaces random variables are usually defined on, such as the unit interval with Lebesgue measure, do not allow for all the construction one wants to make (an uncountable family of independent random variables for example). In order to make all the constructions one wants to work with possible, one needs to work with more esoteric tools from measure theory. The problem is even larger when one turns to stochastic processes or adapted stochastic processes.

For this reason, people have worked on probability theory from the model theoretic view, which gives answers to existence questions much closer to the categorial view. A relatively readable introduction to this field is given in the book "Model Theory of Stochastic Processes" by Fajardo and Keisler. Their paper Existence Theorems in Probability might also be of interest.

-
I should have added a caveat to my answer that working with stochastic processes forces one to grapple with probability spaces more. But I've never heard of anyone actually wanting to consider an uncountable family of independent random variables. –  Mark Meckes Apr 9 '10 at 13:19
They actually occur in mathemtical economics. One wants to have a continuum of agents to apply analysis and one wants to be able that their independent actions cancel out in the aggregate. One wants a law of large numbers for such cases. Finding spaces on which one can make this work turned out to be hard but possible. –  Michael Greinecker Apr 9 '10 at 13:36

I want to post the following as a comment on many of the answers and comments already given.

Several people have said, "Well, watch out -- probability theory is not really the study of probability measures, but rather the study of certain quantities preserved under certain equivalence relations on probability measures, like distribution functions."

I certainly accept this point. In fact, I had more or less accepted it before I asked the question, although I admittedly didn't give much indication of this in the question itself. To be clear, I am aware "rewriting" impulses I have when reading about basic measure-theoretic probability are taking me in a direction away from the material of mainstream probability theory. I have two responses to this:

1) Okay, let's agree that the definition and study of a category of probability spaces is not the domain of probability theory per se. But this does not mean that it's not useful or worth studying.

1a) If this branch of mathematics is not probability theory, what is it? [User "coudy" gave an answer saying that this is ergodic theory. I was unduly dismissive of this answer at first, and I apologize for that. I still don't think that "ergodic theory" is exactly the answer to my question, for instance because so far as I understand the subject it focuses almost exclusively on the dynamical aspects of iterating a measure-preserving transformation of a probability space. (By way of analogy, the branch of mathematics that studies the category of finite type schemes over a field $K$ is arithmetic geometry, not arithmetic dynamics.)

1b) While I agree that probability theory is at present not concerned with such structuralist questions, is it clear that it shouldn't be? Or, in less polemical terms, is there no advantage or insight to be gained by studying the structural aspects of probability spaces?

2) I think an outsider to probability theory has a right to ask: "Okay, if probability spaces are really not the point of probability theory, why do they appear so prominently in all (so far as I know) modern foundations of the subject? Wouldn't -- or couldn't -- it be better to isolate exactly the structure that probability theory actually does care about and study this structure explicitly from the outset?"

By way of analogy, consider the notion of a "differentiable atlas" in the study of smooth manifold theory. Gian-Carlo Rota referrred to atlases as a polite fiction, meaning (I think) that they are present in the foundations of the subject but do not really exist in the sense that the practitioners of the subject do not think about them and ask questions about them. They don't do any harm so long as you don't take them very seriously, but I have seen students get caught up on this point and "ask too many questions". The more modern approach of a structure sheaf seems like an improvement here -- it does the same work as an atlas but is something that the practitioners of the subject actually care about, so it is not at all a waste of time to "think deeply about structure sheaves". Indeed, the concept of "structure sheaf" is incredibly prevalent in other areas of mathematics, to the extent that if you are founding a new branch of geometry, knowing about structure sheaves will ease the birthing process.

So the dual question to 1) here is "What is the kind of mathematical structure that probability theorists are interested in studying?" (Happily, many of the very nice answers above do in fact address this question.)

-
Unfortunately I didn't see this nice answer/commentary until just now. A propos of your question 2), you might be interested to look at the classic two-volume probability text by Feller, in which probability spaces play a surprisingly small role and are not even introduced until well into volume 2. –  Mark Meckes Sep 4 '12 at 15:27

A category consists in a class of objects together with a class of morphisms. Measure theory together with morphisms between measure spaces is the topic of ergodic theory. So if you are interested in a categorical viewpoint at measure theory, just take a look at advanced books on ergodic theory.

Now some references. Glasner's book "ergodic theory via joinings" is probably what is close to a full blown categorical account of some basic concepts in ergodic theory. Rudolph's "Fundamentals of measurable dynamics: ergodic theory on Lebesgue spaces" is also pretty geared toward such an account. If you are interested in applications of ergodic theory to Lie group actions and diophantine approximation, you should consult the appendices in the books of R. Zimmer "ergodic theory and semisimple Lie groups". These appendices summarize the categorical results relevant to these questions.

Note however that most books on ergodic theory are pretty quick on the categorical stuff. Ergodic theory is a subject which is of interest to group theorists, dynamic people, probabilists, combinatorists, physicists, computer scientists,... So, really, it makes no sense to spend too much time on some fundational material that is irrelevant to most of these people, and to most applications.

In contrast to algebraic geometry, which is built like a cathedral, and for which category is a very interesting foundational material, ergodic theory is more like of a bazaar. Its structure is definitely transverse to the usual classification of mathematics (algebra, analysis, geometry), and even transverse to the classification of science (math, physics, computer science, biology) you may be accustomed to. Much of the steam in ergodic theory comes from the many interactions between these communities. It is absolutely crucial to keep the entrance level as low as possible to get as much people as possible on the boat. Putting forward a categorical approach in the textbooks or in conferences would do much harm to the field.

The references I provide should answer your four questions. Let me just add a comment. If you define a Borel space as a set endowed with a $\sigma$-algebra, you will soon run into many problems (e.g. a morphism at the level of the algebras not necessarily comes from a map between the sets, also a non-Borel non-Lebesgue measurable subset of $[0,1]$ endowed with the Lebesgue measure is a perfectly well defined measure space, and you definitely don't want it ), so that's why people don't usually define it that way. There are two choices in use at the moment: the Borel standard spaces, and the Lebesgue spaces. I am on the second wagon but it would be too long to explain why.

-
Pete, I think you were too quick to dismiss coudy's answer (and frankly, I don't see how you found it disrespectful). Many results and methods in probability theory can be rephrased in terms of ergodic theory, which means this is a perfectly on-topic response to your question. –  Tom LaGatta May 24 '10 at 1:14
@Tom LaGatta: I agree with you, and awhile ago I deleted these comments and removed my downvote. In some sense coudy's answer is the closest I have received so far to one of the aspects of the question, although I still maintain that it is not quite dead-on. I have explained this in more detail in my CW answer below. –  Pete L. Clark Dec 19 '10 at 10:44

There is an early paper by Victor Bogdan called "A new approach to the theory of probability via algebraic categories" (#54 here) which may be of interest.

-

Misha Gromov, "In a Search for a Structure, Part 1: On Entropy." http://www.ihes.fr/~gromov/PDF/structre-serch-entropy-july5-2012.pdf provides some interesting category-theoretic musings, among other things. One curious 'other thing' is that the Fisher metric is the flat metric on complex projective space.

-

Last year Voevodsky has given a talk at MIAN about his approach to probability theory; there is online a videorecording in Russian. I do not know if anything is written on this.

There was also an old Russian book (in Russian, afaik not translated, from the 70s) developing a somewhat similar approach but I do not quite remember the reference. I could look for it, though, if there is interest...

-
I don't know what is possible after 2 years - but if you're still here, I am interested –  Ilya Mar 31 '12 at 20:08

Anyway I find "Bichteler :Integration, Springer LNM 315"

it is about the foundation of the theory, the style is similar to Bourbaki, and may be adaptable for a categorical view.

-