Why is $\mathbb R^{\mathbb N}$ not high-dimensional enough?

Question

In this paper [1], the authors consider the limiting distribution of $$S_{n,p}:=\frac{1}{\sqrt n}\sum_{k=1}^nX_k$$ for $p\rightarrow\infty$ as $n\rightarrow\infty$, where $X_1, X_2,\dots, X_n$ are centered iid $\mathbb R^p$-valued random variables. Here $p$ is a positive integer, possibly depending on $n$. They argue that there will be no limiting distribution of $S_n$ and write

Of course, we can think of $S_n$ as random elements in $\mathbb R^{\mathbb N}$ (the countable product of $\mathbb R$), but weak convergence in $\mathbb R^{\mathbb N}$ is equivalent to finite dimensional convergence, which is too weak a result to develop genuinely high-dimensional inference methods.

What do they mean? I understand that weak convergence (= convergence in distribution) in $\mathbb R^{\mathbb N}$ is equivalent to weak convergence of the finite dimensional marginals. But I don't get why (or in what sense) $\mathbb R^{\mathbb N}$ is not high-dimensional enough.

The authors continue to consider sequences, so I doubt that taking the countable product is the issue here.

Edit: It seems to be that there is a misunderstanding on my part (see comments to the answer below). My thoughts are rooted in the following considerations:

• If we let $p\rightarrow\infty$, we eventually have that $p = \infty$. In this case, we are in $\mathbb R^{\mathbb N}$.

• Let $\mathcal B$ denote the Borel $\sigma$-algebra over $\mathbb R^{\mathbb N}$ (endowed with the topology of point-wise convergence). In addition, let $\mu\pi_p$ denote the pushforward probability measure of some probability measure $\mu$ on $\mathcal B$ under the coordinate projection $\pi_p$ mapping $\mathbb R^{\mathbb N}$ into $\mathbb R^p$.

• In the space of probability measures on $\mathcal B$, we consider the sequence $(P_n)_n$, where $P_n$ is the distribution of $$S_{n,\infty} :=\frac 1{\sqrt n}\sum_{k=1}^{n}\boldsymbol X_k$$ with $\boldsymbol X_1, \boldsymbol X_2,\dots, \boldsymbol X_n$ being centered iid $\mathbb R^{\mathbb N}$-valued random variables.

• I can show that finite-dimensional weak convergence is equivalent to infinite-dimensional weak convergence in the sense that a sequence of probability measures $(P_n)_n$ on $\mathcal B$ converges weakly to a probability measure $P$ on $\mathcal B$ if and only if the sequence probability measures $(P_n\pi_p)_n$ converges weakly to the probability measure $P\pi_p$ for all $p\in\mathbb N$. This is Example 2.6 in [2].

• In my case, $P_n\pi_p$ is the distribution of $S_{n,p}$ for any fixed $p\in\mathbb N$. I can show that $S_{n,p}$ converges weakly to $P\pi_p$, which is a $p$-variate normal distribution. Since finite-dimensional convergence is equivalent to infinite-dimensional convergence (in the above sense), I conclude from the convergence of $(P_n\pi_p)_n$ that $(P_n)_n$ converges weakly to $P$, and $P$ must be a Gaussian process as a Gaussian process is characterized by its finite-dimensional marginal distributions (which are multivariate normal distributions).

• Conclusion:

  • I can model situations in which $p>n$ by considering $\mathbb R^{\mathbb N}$-valued random variables.
  • The distribution, as $n\rightarrow\infty$, of a sum of $n$ iid copies of such random variables scaled by $\frac 1{\sqrt n}$ can be assessed by employing known theory for weak convergence in finite-dimensional spaces.

  That is, we have a model for $p>n$ situations with the (easy and known) theory from the finite-dimensional case. This sounds great to me, but apparently I am missing a key point here.

My question is now: why is this not sufficient? Or is there an error in my line of thinking, and if so, where? In other words, if I suggested the above model for $p\rightarrow\infty$ as $n\rightarrow\infty$, why wouldn't it be considered truly high-dimensional (as indicated by the quote)?

My initial thought was that $\mathbb R^{\mathbb N}$ might be not high-dimensional enough in some sense, hence the question title. However, @Iosif Pinelis points out that $\mathbb R^{\mathbb N}$ is rather "'too high' dimensional for the finite-dimensional convergence to work." Why should the finite-dimensional convergence not work?

---

[1] Victor Chernozhukov, Denis Chetverikov, Kengo Kato, and Yuta Koike (2023). High-Dimensional Data Bootstrap

[2] Patrick Billingsley (1969). Convergence of Probability Measures

Answer 1

The authors did not say anything like "$\mathbb R^{\mathbb N}$ is not high-dimensional enough."

Rather, they said

"finite dimensional convergence [...] is too weak a result to develop genuinely high-dimensional inference methods".

The meaning here is rather the opposite: $\mathbb R^{\mathbb N}$ is "too high" dimensional for the finite-dimensional convergence to work. For instance, if you have a sequence $((Y_1(p))_{p\in\Bbb N},(Y_2(p))_{p\in\Bbb N},\dots)$ of stochastic processes of which you only know that the finite-dimensional distributions of the process $((Y_n(p))_{p\in\Bbb N}$ converge to the finite-dimensional distributions of some process $((Y_\infty(p))_{p\in\Bbb N}$ as ($n\to\infty$), you cannot in general conclude anything about convergence of the distributions of a functional of the entire path of the process $Y_n(\cdot)$ to the distribution of that functional of the entire path of the process $Y_\infty(\cdot)$. An example of such a functional is $\Bbb R^{\Bbb N}\ni y\mapsto\sup_{p\in\Bbb N}y(p)/a(p)$ for some positive function $a$ on $\Bbb N$.

So, if one wants to study the behavior of $S_{n,p}:=S_n$ for large $n$ and $p$, then it is not enough to know the behavior of $S_{n,p}$ for large $n$ but only for a fixed finite set of values $p$. You already "understand that weak convergence (= convergence in distribution) in $\mathbb R^{\mathbb N}$ is equivalent to weak convergence of the finite dimensional marginals." So, the highlighted thesis follows.

---

On a somewhat positive note: In this comment, the OP wrote:

The fact that finite dimensional convergence implies infinite dimensional convergence sounds like a nice feature in that regard (as it simplifies things by a lot). In the paper, and in your answer, this fact sounds like I huge disadvantage though.

Of course, I said nothing of this sort. In fact, I did not talk about any advantages or disadvantages at all.

What can actually be said on this matter is the following. The convergence of the finite-dimensional distributions is of course necessary for the convergence of the distributions of the entire processes. Moreover, there are a number of results saying that, with the additional tightness condition, the convergence of the finite-dimensional distributions is also sufficient for the convergence of the distribution of the entire processes -- see e.g. Theorems 7.1 and 13.1 in Billingsley.

Furthermore, the tightness condition cannot be dropped -- cf. Example 2.7 in Billingsley's book.

Yet more, according to Prokhorov's theorem, the tightness condition is necessary if the paths of the processes are in a Polish space.

Answer 2

Often in mathematics, especially in applied mathematics, we have some finite object we care about, and we choose to study it by putting it in a sequence and studying the limit of the sequence, which is often easier to calculate with. The assumption that knowledge about the limit translates to useful knowledge about a fixed term in the sequence is heuristic, but it might be possible to make this rigorous.

To do this, one has to define a mathematical notion of what error is acceptable for a fixed term in the sequence, and check that the error arising from the approximation by the limit goes to $0$ as expected for the relevant notion of limit. But then one also has to check that the convergence to $0$ is fast enough (at least for cases that correspond to data that appears in the real world) that the error for realistic fixed values of $n$ is actually small.

What you've done in your argument is, without specifying what forms of error we care about in the finite $p$, $n$ model, assumed that taking the limit as $p,n$ go to $\infty$ in the weak convergence topology produces an approximation with acceptably small error. This might be true, but it's certainly not a mathematically rigorous argument step, you have just assumed it. You conclude that the known theory in the case where $p$ is fixed and we take the limit as $n$ goes to $\infty$ suffices.

The authors instead start from the assumption that the theory from the case where $p$ is fixed and $n$ goes to $\infty$ is applicable only to the case of $p$ small relative to $n$. This might be justified by empirical observations (of failures of previous methods?) or mathematical arguments elsewhere in the paper, but on its own is just as rigorous as your argument since there is no definition of "good enough" for fixed $p,n$ to use. They conclude by the contrapositive that the topology of weak convergence does not suffice to ensure limits of sequences are a good approximation of the members of the sequence and suggest an alternative topology.