Which limit to take as a key applied math decision

Question

The Borel-Kolmogorov paradox refers to situations where non-uniqueness in the notion of conditioning on a set of measure zero leads to apparent contradictions. As a formal matter, one requires instead to condition on "the" generating sigma algebra, which vanquishes non-uniqueness by fiat. For a technical explanations see this paper. Billingsley's measure theory book has a nice treatment as well.

I am looking for examples where a formal non-uniqueness was resolved by applied considerations which suggested a natural "tie-breaker".

A simple example from this paper illustrates the idea. Let $X$ and $Y$ be independent standard normal random variables. What is the conditional distribution of $X$ given that you are on the (measure-zero) line where $X = Y$? The answer will vary depending on if you condition on $Z_1 = 0$ where $Z_1 \equiv X - Y$ or $Z_2 = 1$ where $Z_2 \equiv X/Y$, to give just two of an infinite number of examples. So in a given situation which $Z$ is the "right" one to use?

My question is not about the Borel paradox or modeling random phenomena per se.

I am interested broadly in hearing about situations where

• we have a mathematically well defined condition ($x = y$ as above)
• we want to study some applied model when that condition is satisfied
• the conclusions we reach will differ depending on the way we approach (as taking a limit) that condition formally

Finally I am interested in how this ambiguity is resolved by "physical" considerations.

I would make the problem sharper if I could, but the reason I want examples is precisely to help focus my thinking. I find it intriguing that it is not enough to have a well defined condition and a well defined model, one must also justify (by way of interpretation) which limit to take!

I anticipate there are many examples from physics of which I am unaware and perhaps some from the literature on finite elements for solving PDEs.

(Apologies for the pay-wall links.)

Answer 1

I found the Marginalization Paradox(Wallstrom) from Bayes theory to be an excellent example explaining how practical motivation guided people to choose one of two possible formalization of improper priors(in which kind of limit we define improper priors) in Bayes modeling.

Suppose that $p(x\mid\theta)$ is the density function a Lebesgue dominated model on a Polish space and $$\pi(\theta\mid x)=\int_{\Theta}p(x\mid\theta)\pi(\theta)d\theta$$ is the posterior w.r.t. (improper) prior $\pi(\theta)$ by Bayes principle. For an improper prior $\pi(\theta)$ defined as a limit of proper prior sequence $lim_{n}\pi_{n}(\theta)$, it is usually chosen to be some kind of limit.

Then for each $\pi_{n}(\theta)$ and a collection of observation $x$ we have corresponding posterior $\pi_{n}(\theta\mid x)$. According to Bayes principle, it seems that $\pi(\theta\mid x)=lim_n\pi_{n}(\theta\mid x)$ and $\pi(\theta\mid x)=\int_{\Theta}p(x\mid \theta)lim_{n}\pi_{n}(\theta)d\theta$ and probability limit in terms of marginal posterior density are all acceptable. However the inference based on pointwise limit and probability limit are completely different.(Wallstrom) This caused trouble when we try to make inference based on marginal density of the observations.

From the earliest time, people tend to choose a pointwise limit definition of improper prior. But later studies showed that such a limit will lead to frequentist inconsistency in sense that the posterior inference will actually deviate from the "TRUE" model with strictly positive probability. Stone showed in 1980s that such a paradox can be solved if we require the posterior to be defined as probability limit instead of pointwise limit.This discovery actually solidify the foundation of Bayes theory after Freedman and Diaconis's criticism of inconsistency of Bayes inference in 1970-1980s. In some sense it also facilitated later computational breakthrough by Gelfand and Smith around 1990s.(Kass&Wasserman)

But in practice, the probability limit usually does not exist so people will still choose to use pointwise limit if they know such a definition will NOT cause internal inconsistency(for example single-mode posterior). In fact, I knew some researchers who spent equal amount of time in building model and justification of using an informative prior(sometimes with a very specific Kullback-Leibler support) because they do not want to discuss about inconsistency. This is probably the elephant in the room in today's applied Bayesian community which will prevent further development if not solved appropriately.