Suppose we sample $k$ vectors $v$ from normal distribution centered at zero and diagonal covariance with diagonal entries $1,\frac{1}{2},\ldots,\frac{1}{d}$ and normalize $v$:
$$\frac{v_1}{\|v_1\|},\frac{v_2}{\|v_2\|}, \ldots ,\frac{v_k}{\|v_k\|}$$
We apply Gram-Schmidt orthogonalization to this list to get a new list of vectors:
$$u_1,u_2,u_3,\ldots,u_k$$
$u_i$ contains component of $v_i$ orthogonal to $v_{i-1},v_{i-2},\ldots$ so we expect it to shrink with $i$. Can we say how the expected value of $\|u_i\|^2$ decays with $i$? For isotropic Gaussian, it decays linearly, while simulation below suggests there is a simple correspondence for the case at hand, can we say what it is?
For $d=1000$, the following is graph for mean and variance of $\|u_k\|^2$
Motivation some optimization tasks in ML exhibit $1/k$ decay of eigenvalues (Section 3.5 of NQM paper. $1/k$ decay is implied by log-regularity, and by alpha-capacity conditions, Appendix A.1 of Varre paper. Knowing the rate at new vectors approach linear dependence lets us bound regret from using a subset of the full dataset for linear estimation problems
