22
$\begingroup$

One can construct a category of probability spaces, but this category has no products. Now probability theory relies strongly on the ability to build independent products, the product measure. In a sense, the notion of independence is what distinguishes probablity theory from the theory of finite measures.

Is there a categorial way to make sense of and enlighten the notion of independent products in category theory?

It is possible to formulate independence in Lawvere's category of probabilistic mappings (Borel spaces as objects and Markov kernels as morphisms) in terms of constant morphisms, but I think this is not very enlightening, conditional independence is built into the morphisms. Maybe, this is what one has to do when putting probability center stage?

I do know the rudiments of categry theory, but I would prefer an answer that does not require too much immersion in category thory, provided that is possible.

$\endgroup$
4
  • 1
    $\begingroup$ A slight aside. My impression, given what I'm presently working on, is that this is the approach of how people are thinking about Mertens-Zamir universal type structure. $\endgroup$ Feb 5, 2013 at 22:42
  • 1
    $\begingroup$ @Rabee Tourky: My impression is that most of this stuff can be done in terms of Markov kernels. Grabiszewski constructs his type spaces using kernels directly. The projective limit results underlying the Mertens-Zamir type construction of the universal type space as the space of coherent hierarchies can be proven by getting kernels as disintegrations and then applying the Ionescu-Tulcea theorem. The probability-valued functions used by Heifetz and Samet in their construction are equivalent to kernels... $\endgroup$ Feb 5, 2013 at 23:09
  • 6
    $\begingroup$ It is a monoidal structure, and probably cannot be recovered from the plain category. $\endgroup$ Feb 6, 2013 at 0:29
  • 3
    $\begingroup$ I agree with Martin; I think it is reasonable to think of product measure as analogous to the tensor product of noncommutative rings (which is neither the categorical product nor the categorical coproduct in the category of rings). $\endgroup$ Feb 6, 2013 at 4:33

2 Answers 2

18
$\begingroup$

Very nice question! ;) I wrote a short paper about this question about ten years ago, see http://arxiv.org/abs/math/0206017 (My apologies for advertising my own work, but this is exactly the question I asked myself at that time).

The product of probability spaces is tensor product in the sense of category, as Martin Brandeburg also pointed out in a comment. But it has an additional structure, you have natural morphisms onto the factors in the tensor product. This is because the projections onto the two factors that you have for the Cartesian product (of sets) preserve the measures, so they are also morphisms in the category of probability spaces. I called this structure a tensor product with projections: for two objects $\Omega_i=(\Omega_i,\mathcal{F}_i,P_i)$, $i=1,2$, you get $\Omega_1\otimes\Omega_2=(\Omega_1\times\Omega_2,\mathcal{F}_1\otimes\mathcal{F}_2,P_1\otimes P_2)$ and random variables $X_i:\Omega_1\otimes\Omega_2\to \Omega_i$, $i=1,2$.

You can use this "tensor product with projections" to characterise independence of random variables: two r.v. $Y_i:\Omega\to\Omega_i$, $i=1,2$, defined on the same probability space $\Omega$, are independent iff they factorise, i.e., if there exists a r.v. $Z:\Omega\to\Omega_1\otimes \Omega_2$ such that $Y_i=X_i\circ Z$, $i=1,2$.

The notion dualises to the algebras of functions on a probablity space, where it becomes a tensor product with inclusions. Generalising to not necessarily commutative algebras, it includes notions of independence used in noncommutative (or quantum) probability, like the freeness.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you, this seems to be exactly what I'm looking for! I will wait a bit and check your paper before accepting your answer. $\endgroup$ Feb 6, 2013 at 9:53
7
$\begingroup$

One resource you may like is this recent paper by Culbertson and Sturtz on A Categorical Foundation for Bayesian Probability.


Here are some thoughts on the category $\mathrm{Meas}$ of measurable spaces, where the objects are sets equipped with $\sigma$-algebras, and morphisms are measurable functions (i.e., random variables). This seems like the most natural place to play with foundations of probability, since the category $\mathrm{Meas}$ has a natural concept of a tensor product, where the corresponding $\sigma$-algebra is generated by product sets.

  • Question: Is $\mathrm{Meas}$ a Cartesian closed category under the tensor product?

A measure is a countably-additive function defined on $\sigma$-algebras. Typically we learn that measures take values in the real numbers (a Borel-measurable space), but there is no reason that they cannot take values in more general structures, e.g., vector spaces or topological groups. To the best of my knowledge, the most general formulation is due to Tarski, whose monoids are valued in commutative monoids. To Tarski, a "measure" is really a functor from a subcategory of $\mathrm{Meas}$ to the category $\mathrm{ComMon}$ of commutative monoids. This is some universal object, and all other measures can be derived from it. I'm still iffy on this approach, but understanding the Tarski functor is the current thesis project of Tyler Bryson, a masters student at the Courant Institute. He should have more details in a few months.

To define independence, we need to multiply probabilities, so measures should take values in a ring. This multiplicative structure is also important for conditional probabilities, more generally, as well as integration theories. If the space admits symmetries (i.e., a group action), then Tarski's approach spits out a natural multiplicative structure, so I think we're safe in general.

Next,, you can normalize all the measures to have size $1$, turning them into probability measures, but I recommend against it from the categorical point of view. In statistical physics (and more generally, statistics), these normalization constants are the hardest things to compute. I recommend keeping track of them combinatorially, then reducing them at the end. Moreover, it may be useful to keep track of a few different "scales" of measurements, where it is not clear which one to normalize to $1$. This is even seen in the case of vector-valued measures. In the quantum setting of nonnegative-operator-valued measures, a uniform scaling can be chosen, but again the normalization constants are difficult spectral integrals to compute.

No big deal if you want probability measures in the end, just make sure that $1$ makes sense to use in the context you're studying.

Anyway, let $R$ be a topological commutative ring and let $M(X) := M(X,R)$ denote the space of $R$-valued measures on $X$. This is a measurable space, when equipped with the minimal $\sigma$-algebra so that the evaluation map is measurable. This $M$ is an endofunctor on $\mathrm{Meas}$, and closely related to the Giry monad.

Consider the tensor product $XY := X \otimes Y$ of measurable spaces. There are natural projection maps $\pi_X : XY \to X$ and $\pi_Y : X \otimes Y \to Y$. Measures push-forward, so there are natural maps $(\pi_X)_* : M(XY) \to M(X)$ and $(\pi_Y)_* : M(XY) \to M(Y)$. Note that $M(XY)$ corresponds to the joint distributions over $X$ and $Y$, and the resulting projections the marginal distributions.

Note that $M(XY)$ corresponds to the joint distributions over $X$ and $Y$, and the resulting projections the marginal distributions.

There is also the natural space $M(X)M(Y) := M(X) \otimes M(Y)$, and its corresponding projection maps $\pi_{MX} : M(X)M(Y) \to M(X)$ and $\pi_{MY} : M(X)M(Y) \to M(Y)$.

As long as there is some Fubini-type theorem present, there should be a natural map $\varphi : M(X)M(Y) \to M(XY)$ sending a pair of measures to their product measure, corresponding to independent random variables. This should correspond to a commutative diagram, but I don't see it.

Again, watch for the normalization constants, since there are two $M$'s in the source of this map but only one $M$ in the target. Dependent random variables then should be quantified by a failure of some diagram to commute.


Even though I didn't answer the question, hopefully you will find some of this content useful.

If you can improve this community-wiki post, please go for it.

$\endgroup$
3
  • $\begingroup$ Thank you! These are certainly interesting things to explore. I am familiar with the paper by Culbertson and Sturtz: mathoverflow.net/questions/117294/… :-) $\endgroup$ Feb 6, 2013 at 9:52
  • $\begingroup$ @Michael Greinecker, haha oops. As you can tell, I'm just synthesizing a lot of these ideas for the first time, and losing track of exactly where they come from. My apologies! $\endgroup$ Feb 6, 2013 at 20:48
  • $\begingroup$ @Tom Don't worry, it's great that you aggregate all this information. $\endgroup$ Feb 7, 2013 at 7:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.