Many results in probability theory/random matrix theory/etc require probability distributions with finite fourth moments; what is the measure of such probability distributions (in the space of probability measures)? While the question "what RVs has finite fourth moment" is answered here, I'm trying to determine how often you would expect "real-life data" to conform to hypotheses in the various results above.

Edit: since this isn't my primary field, my question is naive, apologies. I was primarily interested in the measure of finite moment distributions relative to infinite ones, particularly fourth and lower moments (although I suspect it won't matter). However, thanks to Nate, below, I now know there is no canonical measure on distributions. Since there are no canonical measures, my question becomes:

What are some commonly-used measures for the space of distributions, and what proportion do finite moment distributions occupy in them?

Additionally, because there aren't canonical measures, I'd like to know if these finite moment distributions are dense in two particular topologies, the ones induced by $L_p$ and Fisher metrics (generally divergence-induced metrics). (I haven't though about how the two metrics are related topologically, if at all).

  • 1
    $\begingroup$ "what is the measure" - with respect to which measure on the space of probability measures? There isn't a canonical choice. $\endgroup$ – Nate Eldredge Jul 27 '17 at 14:21
  • 1
    $\begingroup$ There is however a canonical topology on this space - the weak topology. You could ask the same question for category instead of measure: is the set meager / comeager? $\endgroup$ – Nate Eldredge Jul 27 '17 at 14:26
  • $\begingroup$ @NateEldredge You're right, I hadn't included a topology; initially I was thinking of a topology induced by $L_p$ or divergence metrics, but hadn't specified, hoping there might be a "more" canonical choice (by way of common usage, if nothing else). $\endgroup$ – Steve Jul 27 '17 at 14:47
  • $\begingroup$ Perhaps you could edit your question to ask something precise. Or, if part of the question is how to formulate the problem precisely, please ask that clearly. Also, I am not clear from your comments whether you are now interested in measure or topology. $\endgroup$ – Nate Eldredge Jul 27 '17 at 14:57
  • $\begingroup$ @NateEldredge Thanks for the comments, reformulated. $\endgroup$ – Steve Jul 27 '17 at 15:32

I'll answer the topological version. Let's work in one dimension for simplicity, so we consider the space $\newcommand{\PR}{\mathcal{P}(\mathbb{R})}\PR$ of probability measures on $\mathbb{R}^1$.

Most of the commonly encountered metrics on $\PR$ induce the same topology: the weak topology. The standard reference for facts about this topology is Billingsley, Convergence of Probability Measures. The basic definition is that a sequence of measures $\mu_n$ converges weakly to a measure $\mu$ if $\int f\,d\mu_n \to \int f\,d\mu$ for all bounded continuous functions $f : \mathbb{R} \to \mathbb{R}$. However, for our purposes, a convenient fact is that when working on $\mathbb{R}$ (or any separable locally compact metric space), "bounded continuous" can be replaced with "continuous and compactly supported".

I claim the set of measures having bounded support is dense in $\PR$. (In particular all such measures have finite fourth moment.) To see this, fix a measure $\mu \in \PR$ and let $\mu_n(A) = \frac{\mu(A \cap [-n,n])}{\mu([-n,n])}$. Clearly $\mu_n$ is a probability measure with bounded support. Let $f \in C_c(\mathbb{R})$ and choose $N$ so large that $f$ is supported in $[-N,N]$. Then for all $n \ge N$ we have $$\int f\,d\mu_n = \frac{1}{\mu([-n,n])} \int f\,d\mu \to \int f\,d\mu$$ as $n \to \infty$. So $\mu_n \to \mu$ weakly.

On the other hand, the space of probability measures $\newcommand{\PF}{\mathcal{P}^4(\mathbb{R})}\PF$ with finite fourth moment is meager in the weak topology. Let $A_k$ be the set of all probability measures having (uncentered) fourth moment $\le k$, so that $\PF = \bigcup_n A_k$. I claim $A_k$ is closed and nowhere dense.

Let $\mu_n \in A_k$ and suppose $\mu_n \to \mu$ weakly. Set $f_j(x) = \min(x^4, j)$ so that $f_j$ is bounded and continuous. We have $\int f_j\,d\mu_n \le \int x^4\,d\mu_n \le k$ for all $j,n$. Letting $n \to \infty$ we have $\int f_j\,d\mu = \lim_{n \to \infty} \int f_j\,d\mu_n \le k$. Now letting $j\to \infty$, we have $\int x^4\,d\mu = \lim_{j \to \infty} \int f_j\,d\mu$ by monotone convergence, so $\int x^4\,d\mu \le k$ and we have $\mu \in A_k$. Thus $A_k$ is closed.

Suppose $\mu \in A_k$ and let $\mu_n = (1 - \frac{1}{n}) \mu + \frac{1}{n} \delta_n$ where $\delta_n$ is a point mass at $n$. An argument like the above shows that $\mu_n \to \mu$ weakly, but you can also see that $\int x^4\,d\mu_n \ge n^3$, so for sufficiently large $n$ we have $\mu_n \notin A_k$. Thus $\mu$ is not in the interior of $A_k$; $\mu$ was arbitrary so $A_k$ is nowhere dense.

  • $\begingroup$ Coming back from lunch to a beautifully written answer. Thanks, Nate! $\endgroup$ – Steve Jul 27 '17 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.