5
$\begingroup$

This question is partially inspired by the following MO post: What are some of the surprising results of finite sample statistical estimation? and current heated research front of high dimensional statistics.

Instead of asking about surprising results in high dimensions, I will ask what kind of results that holds in low dimensions fails to hold in a higher dimension. And what is its relation with other mathematical branch?

(By high dimensional statistics we usually refer to a high dimension covariate space instead of response space. For example, in a regression setting $Y=f(X)$ we tend to say a problem is of high dimension if $\dim\mathcal{X}\gg \dim\mathcal{Y}$)

For one simplest example, we know that James-Stein estimator performs better than maximum likelihood estimator in terms of $L^2$ norm when the dimension $\dim\mathcal{X}=d\geq 3$; and it turns out to be an equivalent statement that a symmetric random walk in $\mathcal{X}=\mathbb{R}^d$ is transient when $d\geq 3$ via an infinitely divisible stochastic process. Another example is provided in the answer below.

Are there other such examples that can relate high dimensional phenomena in statistics?

$\endgroup$
3
$\begingroup$

A great example that I have in mind is the concentration phenomena in high dimensions. Consider the simplest multivariate normal distribution $X\sim N_d(0_d,I_d)$, we can compute its $L^2$ norm $\sum_iX^2_i=:\|X\|^2\sim\chi^2(d)$. and $X_i^2\sim \chi^2(1)$ independently. With central limit theorem applied on each component, we have that $$\frac{1}{d}\sum^d_{i=1}X^2_i=\frac{1}{d}\|X\|^2\overset{P}{\rightarrow}N_1(1,\frac{2}{d})$$ as $d\rightarrow\infty$. Use the delta method we can see that $\|X\|\overset{P}{\rightarrow}\sqrt{d}N_1(1,\frac{1}{d})=N_1(\sqrt{d},1)$ and therefore we can actually assert that as dimension $d=dim\mathcal{X}\rightarrow \infty$ the random vectors are concentrated around a sphere.

Even more surprising is that if we have another independent $Y\sim N_d(0_d,I_d)$, then we can compute the distribution of $\frac{X\cdot Y}{\|X\|\|Y\|}$ as $d\rightarrow \infty$ is $N_1(0,d)$(multidimensional CLT and delta method) and the distribution of $\|X-Y\|$ as $d\rightarrow \infty$ is $N_1(0,2d)$. These two results claimed that as $d\rightarrow \infty$ two random vectors are most likely to be orthogonal and evenly distributed on the sphere, which is not expected when $d=1,2$.

I really hope to know if there are more such examples with a motivation from consideration of the difference between the geometry of high and low dimensional spaces.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.