Suppose we sample $k$ vectors $v$ from normal distribution centered at zero and diagonal covariance with diagonal entries $1,\frac{1}{2},\ldots,\frac{1}{d}$ and normalize $v$:

$$\frac{v_1}{\|v_1\|},\frac{v_2}{\|v_2\|}, \ldots ,\frac{v_k}{\|v_k\|}$$

We apply Gram-Schmidt orthogonalization to this list to get a new list of vectors:

$$u_1,u_2,u_3,\ldots,u_k$$

$u_i$ contains component of $v_i$ orthogonal to $v_{i-1},v_{i-2},\ldots$ so we expect it to shrink with $i$. 
Can we say how the expected value of $\|u_i\|^2$ decays with $i$? For isotropic Gaussian, it decays [linearly](https://stats.stackexchange.com/a/583564/511), while simulation below [suggests](https://www.wolframcloud.com/obj/yaroslavvb/newton/forum-decay-of-orthogonal-contributions.nb) there is a simple correspondence for the case at hand, can we say what it is?

For $d=1000$, the following is graph for mean and variance of $\|u_k\|^2$

[![enter image description here][1]][1]

**Motivation** some optimization tasks in ML exhibit $1/k$ decay of eigenvalues (Section 3.5 of [NQM paper](https://arxiv.org/abs/1907.04164). $1/k$ decay is implied by log-regularity, and by alpha-capacity conditions, Appendix A.1 of Varre [paper](https://arxiv.org/abs/2102.03183). Knowing the rate at new vectors approach linear dependence lets us bound regret from using a subset of the full dataset for linear estimation problems

  [1]: https://i.sstatic.net/lCnot.png