3
$\begingroup$

This is partially inspired by this question and this blog post.

When trying to express classical probability in the "free probability" setting one takes an algebra of random variables equipped with the expectation as a replacement for a (localizable / pointfree) measure space (more precise statements about this relationship are stated here (ignoring the measure) and here e.g.).

While even in the case including the measure/expectation we may get a nice equivalence / adjunction, it seems to me that doing probability theory in this "free" setting is not without its problems: replacing the measure space $X$ with $L_\infty(X)$, in practice, has the glaring issue that we can't "directly" talk about variables which are not essentially bounded without using the equivalence to pass back to the measure spaces; same with the algebra of random variables with finite moments (I'm thinking of Cauchy-distributed variables as a counterexample).

One naive way of fixing this is looking at the space of all random variables instead. I have two questions (and hope that is not to much for just this post):

  1. Are there obvious reasons why this idea is fundamentally flawed in some way?
  2. If not, has anyone explicitely worked something out in this direction? By this I mean: defining appropriate categories (the space of all random variables is not even normed anymore), functors and proving that they are adjoint / an equivalence (up to duality)
$\endgroup$
  • $\begingroup$ Suggestion: add the oa.operator algebras tag ? $\endgroup$ – Yemon Choi Jul 27 '17 at 21:36
  • $\begingroup$ @YemonChoi Done. $\endgroup$ – Stefan Perko Jul 27 '17 at 21:37
  • $\begingroup$ A relevant reference: Free Convolution of Measures with Unbounded Support, by Bercovici and Voiculescu, Indiana University Mathematics Journal Vol. 42, No. 3 (Fall, 1993), pp. 733-773 jstor.org/stable/24897118 $\endgroup$ – Mikael de la Salle Jul 28 '17 at 9:36
7
$\begingroup$

If you're willing to work in the setting of von Neumann algebras, then the algebra of affiliated operators is a good analogue of the algebra of all random variables. You already said that there is a nice relationship between the probability space $(X, \mathbb{P})$ and the algebra of bounded functions $(L^{\infty}(X), \mathbb{E})$ equipped with the expectation. This pair is a special case of a von Neumann algebra $(\mathsf{M}, \tau)$ equipped with a normal, faithful trace. Such an algebra acts naturally on the GNS space $L^{2}(\mathsf{M},\tau)$ and the algebra of affiliated operators consists of certain unbounded operators on the Hilbert space $L^{2}(\mathsf{M},\tau)$. Namely, a closed densely defined operator $x$ is affiliated with $\mathsf{M}$ iff when we write $x=u|x|$ (the polar decomposition) then $u\in \mathsf{M}$ and all spectral projections of $|x|$ belong to $\mathsf{M}$. The remarkable thing is that the set of affiliated operators is an algebra, when endowed with strong sum and strong product. It means that whenever $x$ and $y$ are affiliated then $x+y$ is a closable densely defined operator whose closure is affiliated with $\mathsf{M}$; the same works for the product. On this algebra we can define convergence in measure, and with this topology the algebra of affiliated operators becomes a complete topological $\ast$-algebra. Fortunately, the algebra of operators affiliated with $(L^{\infty}(X), \mathbb{E})$ is exactly the algebra of all random variables (defined up to a set of measure zero). Therefore it looks like in non-commutative probability the algebra of all random variables is a canonical object built from a pair $(\mathsf{M},\tau)$, exactly as it from a pair $(X, \mathbb{P})$ in classical probability. For more on this topic: E. Nelson, Notes on non-commutative integration, J. Funct. Anal. 15 (1974), 103–116.

I would like to add that this works so nicely only in the case of finite measures; if one wants to work with infinite measures then the functional $\tau$ becomes unbounded and the nice object is the algebra of $\tau$-measurable operators, which corresponds to random variables that are bounded outside a set of finite measure. This is always the nice object and the the set of affiliated operators coincides with the algebra of $\tau$-measurable operators in the case of finite measure.

$\endgroup$
  • $\begingroup$ Unfamiliar with the theory of von Neumann algebras (I'm only sightseeing here) I was completely unaware of this, of course. Interesting construction. $\endgroup$ – Stefan Perko Jul 30 '17 at 7:48
6
$\begingroup$

I want to add some comment on the free probability part of the question. There seems to be a common misbelief that free probability is restricted to bounded operators and can only deal with moments. This is not correct. The notion of freeness has been extended to unbounded operators (though, one needs some "nice" analytic structure, as operators affiliated to a von Neumann algebra, as addressed in the answer by Mateusz); see, H. Bercovici and D. Voiculescu. "Free convolution of measures with unbounded support." Indiana University Mathematics Journal 42.3 (1993): 733-773. In particular, for the free convolution there exists an analytic theory dealing with all probability measures on $\mathbb{R}$, independent of the existence of moments; for an exhaustive theory of this see Chapter 3 of my book https://www.math.uni-sb.de/ag/speicher/publikationen/Mingo-Speicher.pdf

$\endgroup$
3
$\begingroup$

Let me try to answer, what I believe is, the underlying question based upon your two questions. (If your question really is about free probability per se, then this answer is totally irrelevant.) The question arises from studying classical probability theory and trying to integrate it with your knowledge of category theory. (This is essentially the question I asked 15 years ago when I read Lawvere's (1962) seminar notes paper on the Giry monad (monads were yet to be discovered/connected to adjunctions in '62.))

The answer to your first question (1) is no. It's a reasonable idea/viewpoint an we can address classical probability theory (and in particular your example problem, e.g., say half-Cauchy distribution for which the expected value does not exist). To understand the general framework for ''doing'' classical probability theory so you can think in terms of the underlying variables you really need to work in the category of the Giry-algebras. But that's a big step; I think it's easier to start with understanding the Kleisi category of the Giry monad. Then the extension to the Eilenberg-Moore category is a straightforward extension where the basic diagram is the same (only the underlying diagram changes to the category of convex spaces rather than the Kleisi category.) If you have not studied the Kleisi category of the Giry monad I would recommend looking at either Doberkat's work or the papers by Culbertson & Sturtz, and in particular, understand (regular) conditional probability using ''the triangle'' (nLab Giry monad where all the references and a basic explanantion of the Giry monad is given.) Once you grasp the triangle (which consist of the prior and conditional, which composition of those two completes the triangle to give the prior on your ''data space'', the extension from the Kleisi category of the Giry monad to the Eilenberg-Moore category of the Giry monad (or rather its equivalent - the category of convex spaces) is discussed in The factorization of the Giry monad. That work extends the basic foundation given for the special case described by Doberkat with the Giry monad as an endofunctor on the category of Polish spaces, as discussed on the nLab page.

The answer to your second question is then given in 2.

Reply to Stefan's comment (too long for a comment)

This is a great question because when you first start thinking about questions arising in probability theory, it is not at all obvious that the category of Giry ($G-$)algebras (or any category equivalent to it, such as $Cvx$) can answer those questions. Thinking in terms of $Meas^{G}$ is difficult because our intuition provides no guidance, and that category will never be accepted by the scientific/engineering community because of its disconnect between the theory (they are familiar with) and practice; conversely, the problems arising in practice give no (direct) guidance to how to develop the theory further. However, I believe, $Cvx$ is a different story. (The theory of convex spaces is the affine-part of the theory of $K$-(semi)modules. See the dissertation by Meng (see the nLab page metric space where you can download the paper). There is a nice description there; and will help you build an intuitive feeling. Alternatively, if you go to the nLab page you can find the paper by T. Fritz, which has some great examples.)

So how does $Cvx$ help characterize the space of all random variables? Start with the idea that $P(\Sigma A)$, where $A$ is any convex space, is naturally isomorphic to the space of all weakly-averaging affine maps, $\mathbf{P}(\Sigma A) =\{I^{\Sigma A} \stackrel{P}{\longrightarrow} I \, | \, P \textrm{ is }wa \& affine\}$, where $I=[0,1]$ is the unit interval with the Borel $\sigma$-algebra, and natural convex structure. (Here, $P$ and $\Sigma$ are the decomposition of the Giry monad, factorizing through $Cvx$ - the notation used in the paper.) That natural isomorphism $G(\Sigma A) \cong \mathbf{P}(\Sigma A)$ can be viewed in either $Meas$ or $Cvx$ (functor by $\Sigma$ to view it as an iso of $Meas$. (Here I am using the functor $G$ as a functor $Meas \rightarrow Cvx$; not as the endofunctor $Meas \rightarrow Meas$.) This idea was first presented in the arxiv paper Categorical Probability Theory. The first version of that paper just assumed the maps $I^{X} \stackrel{P}{\longrightarrow} I$ were just weakly-averaging and affine. There $X$ was any measurable space. The erroneous proof was pointed out by T. Avery, and in later version I assumed something like the maps $P$ preserve limits of sequences. About 4 years hence I realized that to factorize the Giry monad, you need only look at the counit of that adjunction, i.e., $\mathbf{P} \dashv \Sigma$ - hence it is the subset of weakly-averaging affine maps $\{I^{\Sigma A} \stackrel{P}{\longrightarrow} I \, | \, P \, wa \textrm{ and }affine\}$. [This point is not discussed in the Factorization of the Giry monad paper - the main idea in the paper ($Cvx \cong Meas^{G}$, and the tensorial relationship property, Theorem 9.2, $\mathbf{P}(\Sigma A \otimes_{Meas} \Sigma B) \cong \mathbf{P}(\Sigma (A \otimes_{Cvx} B))$ doesn't require this fact; but for the application we are discussing (random variables) it is relevant.] But I digress... back to the issue at hand. The space $I^{\Sigma A} = Meas(\Sigma A, I)$ is, upon taking the convex space, say $A=[0,\infty)$, the set of all $I$-valued random variables on $A$. Now with these two basic tools (1) The SMCC structure of $Meas$ and $Cvx$, and (2) the relationship between the tensor products, $\otimes_{Meas}$ and $\otimes_{Cvx}$,you can solve most problems arising in probability theory just by playing around with the diagrams to get the ''object of interest'' you require. So, for example, if you have say the measurable map $R_{\infty} \rightarrow I$ mapping $x \mapsto exp(-x)$ then you can exponentiate (apply the functor $I^{\bullet}$) to get a map $I^I \stackrel{I^{exp(-{\bullet})}}{\longrightarrow} {I^{\mathbb{R}_{\infty}}}$. Now if you also have a probability measure on $\mathbb{R}_{\infty}$, the composite gives you the pushforward probability measure on $I$, and (in this special case) it is clear a barycenter exist... On the other hand, you can take the measurable map $exp(-x)$ and apply the functor $\mathbb{P}$ to get an affine map ... This viewpoint of probability leads to a perspective that coincides with practice, and I believe gives a better computational framework for actually ''doing'' (computing) inference maps. That efficiency arises from exploiting the barycenter, i.e., Theorem 9.3. (There are about 10 ways to prove that theorem; I think the proof I gave was less than the best. I'll update that when I develop some further aspects of the theory.

I know this answer is less than the easiest answer, but in the long run I believe it is essential (and the easiest) to answer basic questions and simplify the theory. If you decide to wade through the material and some aspects are not clear feel free to contact me via email. The Bayesian Machine Learning paper has some basic examples, etc., which you might find useful in understanding the Kleisi category. Like I said previously, $Cvx$ turns out to be an easy extension of the Kleisi category of the Giry monad, so its important to understand that basic idea.

$\endgroup$
  • $\begingroup$ There is a lot of information in your paper. So regarding the adjunction induced by the Giry monad, does the functor $\mathsf{Meas} \to \mathsf{Cvx}$ indeed map measurable spaces to the space of all random variables? (btw. what is a convex space?). - Also, can one modify this construction somehow to include measures? (i.e. we replace the category of measurable spaces with a category of measure spaces). $\endgroup$ – Stefan Perko Jul 30 '17 at 7:46
  • $\begingroup$ I edited my original answer (just added a couple of paragraphs) to try and explain just how the Giry algebras can help us answer basic questions arising in prob. theory. $\endgroup$ – kirk sturtz Aug 1 '17 at 8:40
  • $\begingroup$ Thank you. Right now I need to finish my Bachelor thesis, but I will get back to this afterwards. $\endgroup$ – Stefan Perko Aug 6 '17 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.