Interpretation of Bai-Yin theorem and a question about (Hastie, Montanari, Rosset & Tibshirani)

Question

Let $X_n\in \mathbb{R}^{p\times n}$ be a random matrix whose entries are i.i.d. $\mathcal{N}(0,1)$. Define $S_n = \frac{1}{n}X_n X_n^\top$. If $p/n\to y\in (0,1)$, the well-known Bai-Yin theorem yields that with probability 1, we have $$\lim_{n\to +\infty}\lambda_{\mathrm{min}}(S_n)=(1-\sqrt{y})^2\,.$$

In light of this, can we say that there exists $N$ such that for all $n>N$, we have $\lambda_{\mathrm{min}}(S_n)>(1-\sqrt{y})^2/2$ almost surely? This argument was used in Hastie et al. multiple times. For example, on page 38, in the paragraph right after equation (110), the authors wrote "$\lambda_{\mathrm{min}}(\Sigma) \ge c$, and the Bai-Yin theorem (Bai and Yin, 1993), which implies that the smallest eigenvalue of $Z^\top Z/n$ is almost surely larger than $(1 −\sqrt{\gamma})^2/2$ for sufficiently large n".

Let $(\Omega,\mathcal{F},P)$ be the underlying probability space. Define $f_n(\omega)=\lambda_{\mathrm{min}}(S_n)$, where $\omega\in \Omega$. I think that to get $f_n(\omega)>(1-\sqrt{y})^2/2$ for all sufficiently large $n$, we need a stronger type of convergence, i.e., uniform convergence for almost all $\omega$ (i.e., all $\omega$ except a zero probability set), rather than just almost sure convergence (pointwise convergence for almost all $\omega$). In other words, the $N$ should work for almost all $\omega$, rather than that the choice of $N$ depends on $\omega$.

My first question is:

Is the argument in Hastie et al. correct?

I have another simple example. Define $X_n(\omega)=n\omega^n$, where $\omega\in \Omega=[0,1]$ and $P$ is the Lebesgue measure. Almost surely we have $X_n\to 0$. However, it is not true that $X_n(\omega)<1$ holds almost surely for all sufficiently large $n$.

I have another minor question:

What is the underlying probability space in the expression $\Pr[\lim_{n\to +\infty}\lambda_{\mathrm{min}}(S_n)=(1-\sqrt{y})^2]$?

I thought that it might be an infinite product probability space of a univariate Gaussian probability measure $\mathcal{N}(0,1)$. Consider an infinite 2D array $A_{ij}$ whose entries are i.i.d. $\mathcal{N}(0,1)$. Then $X_n$ is just a 2D subarray $\{A_{ij}\mid 1\le i\le p,1\le j\le n\}$. Am I right?

Thank you!

Answer 1

To complete what Lars said: By Theorem II.13 in Davidson and Szarek (2001), $$P(\lambda_{min}(S_n)^{1/2} \notin [1-\sqrt y - t, 1+\sqrt y + t]) \le 2 e^{-nt^2}.$$ Pick, e.g., $t= n^{-1/4}$ to get an event of exponentially small probability (for one fixed $n$, this probability bound is non-asymptotic), and by Borel-Cantelli almost surely only finitely many of these events occur (but again, as Lars said, the rank above which no event occurs is random).

This also answers your question about the choice of the underlying probability space: it does not matter as long as the probability space is rich enough to have a sequence of random matrices $S_n$ such that for all $n$, $n S_n$ has $Wishart(n, I_p)$ distribution. For such probability space, by the Borel-Cantelli lemma and the above bound with $t=n^{-1/4}$, almost surely $\lambda_{min}(S_n)^{1/2} \notin [1-\sqrt y - t, 1+\sqrt y + t]$ for at most finitely many $n$, so that $$ 1 = P\Big( \exists N: \forall n\ge N, |\lambda_{min}(S_n)^{1/2} -(1-\sqrt y)| \le n^{-1/4} \Big) \le P\Big( \lim_{n\to+\infty} \lambda_{min}(S_n)^{1/2} = 1-\sqrt y \Big). $$