# Nonstandard analysis in probability theory

I am quite new at nonstandard analysis, and recently I became aware of its use in probability theory mainly through the following two books:

Although Nelson's book is several decades old, as far as I can see, its approach has not yet caught on. Also, I couldn't find a lot of papers published in the leading probability journals on that topic. I am quite intrigued by that phenomenon. My questions are the following

• Why hasn't nonstandard analysis been widely adopted by probabilists?
• Were there some success stories in some particular sub-fields of probability theory or statistics?
• Does there exist some known fundamental objections in probability theory to the approach in there?
-
The burden of proof is not on probabilists to object to this approach but on this approach to show that it is so much more useful than the standard approach that adopting it will be worth the transition costs. This is a fairly general principle about new ideas, I think. –  Qiaochu Yuan Sep 24 '12 at 2:00
I speculate from Geyer's intro that this approach mainly appeals to a relatively smallish group: statisticians with a desire for rigor and simultaneously very specific applications (stat inference) in mind. Learning this new approach may pay off in licensing the heuristic of thinking about all integrals as sums and all conditioning as division. I do wonder how this approach squares with recent results to the effect that there exist computable joint distributions which yield non-computable conditionals. –  R Hahn Sep 24 '12 at 3:13
I observed (at least at one time) that those working on nonstandard methods in probability would attend conferences and give talks NOT at probability meetings, but at logic meetings. So my impression was: they were not probabilists trying to find new methods to solve their problems; instead they were nonstandardists trying to find places to apply their method. –  Gerald Edgar Sep 24 '12 at 14:02
@Qiaochu Yuan: But NSA is not so much a new idea as a more rigorous way of applying old ideas about infinitesimals. Fourier was already essentially doing Dirac delta functions ca. 1800, Dirac popularized them in 1930, and physicists and engineers generally still think of a Dirac delta function as a spike of infinitesimal width $\epsilon$ and height $1/\epsilon$. Distribution theory came ca. 1935. So if there is a new idea that needs to be proved worthwhile in transition costs, it's distribution theory -- which has never been widely adopted by physicists and engineers. –  Ben Crowell Sep 24 '12 at 22:35
I am clearly biased (and not ashamed of it), but I find it nonstandard that so many people have sharp opinions about nonstandard analysis. –  François G. Dorais Sep 25 '12 at 23:44

Non-standard analysis has been quite successful in settling existence questions in probability theory. Hyperfinite Loeb spaces allow for several constructions that cannot be done on standard probability spaces. In particular, NSA was quite useful for the construction of certain adapted processes. There is a paper by Hoover and Keisler, Adapted Probability Distributions, from 1984, in which the authors show that many of the properties that make hyperfinite Loeb spaces so useful where due to a property they called saturation: A probability space $(\Omega,\Sigma,\mu)$ is saturated if whenever $\nu$ is a Borel probability measure on $[0,1]^2$ and $f:\omega\to[0,1]$ a random variable with distribution equal to the marginal of $\nu$ on the first coordinate, then there exists a random variable $g:\Omega\to[0,1]$ such that the distribution of $(f,g)$ is $\nu$. An example of a saturated probability space that is not a hyperfinite Loeb space is the coin-flipping measure on $\{0,1\}^\kappa$ when $\kappa$ is uncountable. A relatively readable exposition of this approach can be found in the small book Model Theory of Stochastic Processes by Fajardo and Keisler. There are also several related papers and surveys on Keisler's homepage.

In a sense, we nowadays understand fairly well how certain powerful techniques of non-standard analysis work below the surface, so we can use a lot of the constructions freed of NSA. There isn't really anything where it is necessary to use NSA. Still, NSA is a rather powerful and useful tool. A good overview over what it can do for probability theory, mainly the theory of sochastic processes, is in the article by Osswald and Sun in Nonstandard Analysis for the Working Mathematician by Loeb.

-

My colleague Ed Perkins used quite a bit of nonstandard analysis in probability theory in the early 80's. See for example http://www.springerlink.com/content/e636h42166202387/ I don't know if he's used nonstandard analysis more recently. In his lecture notes from the 1999 St. Flour summer school http://www.math.ubc.ca/~perkins/dawsonwatanabesuperprocesses.pdf he remarked "I noticed that some of the theorems were originally derived using nonstandard analysis and I have standardized the arguments ... to make them more accessible. This saddens me a bit as I feel that the nonstandard view, clumsy as it is at times, is pedagogically superior and allows one to come up with novel insights."

Basically I suspect that might be a summary of the general situation. Everything that can be done in nonstandard analysis can be done in standard analysis, and thereby becomes accessible to those who don't know nonstandard analysis. Each method has its advantages and disadvantages, but the big advantage of standard analysis is that it is familiar to more people.

-
@Robert Israel. Thanks a lot for your answer. Your comments have a similarity to Halmos's remarks of nonstandard analysis at en.wikipedia.org/wiki/… –  an12 Sep 24 '12 at 9:39
It's important to remember that nonstandard analysis is standard analysis. It isn't like there are different axioms or anything. One could technically understand "nonstandard analysis" as the use of ultrafilters, but this is a bit like understanding "random variables" as measurable functions. Technically correct, but lacking the correct spirit. –  Kevin O'Bryant Sep 25 '12 at 1:31
But doesn't nonstandard analysis need the Axiom of Choice, or at least need more than just ZF? Of course, some (many?) people include AC within "standard analysis", especially if they include functional analysis, but all the basic "standard analysis" stuff used to do real variable calculus doesn't use full AC, only a weak axiom of countable dependent choice or similar. –  Zen Harper Sep 27 '12 at 1:35

True probabilists have a rather unique way of thinking. It is, if you will allow word-creation, hyper-analytic. This thought pattern seems (anecdotally!) to not be too compatible with algebraic or logical patterns. I'm not talking about basics, of course, but on a high level. I've never met a probabilist who enjoyed or personally valued the theory of modules, for example. I've never met a probabilist who would feel that the model-completeness of algebraically closed fields was super-cool.

If you're not inclined toward such things, then the foundational advantage conferred by NSA is moot. And the intuitive advantages are already exploited without hesitation. In my experience, all probabilists think with NSA ideas by default and without self-conciousness, and without concern about how to "rigorize" the arguments.

To make my point, everyone knows that Brownian Motion is the limit of simple random walks. They don't feel the need to make this rigorous, it is just self evident. That it can be made to be almost trivial using NSA is as interesting as seeing an epsilon-delta proof of continuity. Fine for beginners, but not something for me to spend time on now.

(Edit) Disclaimer: I make no claim to have met a random sample of logicians, algebraists, probabilists, or anyone else. I was at Urbana-Champaign for a number of years, and had classes/seminars with Loeb, Henson, and Burkholder, and am married to a industrial stochastic analyst. I love NSA and find it gorgeous, and I feel the same way about probability (but not stochastic calculus, sorry). I've seen first-hand over almost 2 decades how students and professors react to NSA, but again it was not a random sample.

We all know that essentially every mathematician has a "flavor" or two that they prefer over the others. Some of us are analysts, some algebraists; I love combinatorics, and many others don't give it much respect. All I wanted to point out was that the "flavor" of formal NSA is distinctly different that of today's probability, while "infinitesimal" thinking is already incorporated. This combination, in my humble opinion, is why NSA has not taken hold in probability. There are of course exceptions, with Ed Perkins being the most notable but not the only.

-
The idea that "everyone knows that Brownian Motion is the limit of simple random walks. They don't feel the need to make this rigorous, it is just self evident." really bothers me. Just because there is standard machinery that is cited to show the limit exists does not mean that no one feels the need for rigor. –  BSteinhurst Sep 25 '12 at 2:03
I'm a little bothered by the sweeping nature of the 1st para. I've never met a Banach algebraist who is enraptured by Beck's monadicity theorem, but I'm not sure that proves much. –  Yemon Choi Sep 25 '12 at 7:47
As for 2nd para - well KOB probably has more experience and insight than me here, but this isn't the impression I took (as a non-probabilist, admittedly) from e.g.the Rogers-Williams books on stochastic processes. –  Yemon Choi Sep 25 '12 at 7:52
Well, it was a sweeping question, and I did hedge over and over with "in my experience" and "anecdotally" and "I". I'm aware I'm generalizing like nuts. What I meant, more than a lack of need to be rigorous, is that probabilists (in my meager experience) are comfortable with infinitesimals as rigorous. In light of NSA, that's fine, even if one isn't familiar with the intricacies of internal vs external, saturation, etc. –  Kevin O'Bryant Sep 25 '12 at 20:28
The edit brought forth what I knew you were saying but in a less provocative style. I happen to agree with the main point of your answer, I just got stuck on one particular phrase. –  BSteinhurst Sep 26 '12 at 14:11

The answers given earlier are excellent. I would merely like to supplement them by the observation that the success of NSA and IST in probability and related fields is attested to by the fact that new books continue to be in demand and are being published in this area, in some of the most prestigious series, such as the 2013 book by F. Herzberg entitled "Stochastic Calculus with Infinitesimals", see http://link.springer.com/book/10.1007/978-3-642-33149-7/page/1

-