Suppose we sample $k$ vectors $v$ from normal distribution centered at zero and diagonal covariance with diagonal entries $1,\frac{1}{2},\ldots,\frac{1}{d}$ and normalize $v$:

$$\frac{v_1}{\|v_1\|},\frac{v_2}{\|v_2\|}, \ldots ,\frac{v_k}{\|v_k\|}$$

We apply Gram-Schmidt orthogonalization to this list to get a new list of vectors:

$$u_1,u_2,u_3,\ldots,u_k$$

$u_i$ contains component of $v_i$ orthogonal to $v_{i-1},v_{i-2},\ldots$ so we expect it to be shrink with $i$. 
Can we say how the expected value of $\|u_i\|^2$ decays with $i$? For isotropic Gaussian, it decays [linearly](https://stats.stackexchange.com/a/583564/511), while simulation below [suggests](https://www.wolframcloud.com/obj/yaroslavvb/newton/forum-decay-of-orthogonal-contributions.nb) there is a simple correspondence for the case at hand, can we say what it is?



[![enter image description here][2]][2]

**Motivation** some optimization tasks in ML exhibit $1/k$ decay of eigenvalues (Section 3.5 of [NQM paper](https://arxiv.org/abs/1907.04164). Also Wigner's semicircle law predicts $1/k$ decay for the first few eigenvalues), knowing this rate would let us bound the regret from using batch size $<d$ in each optimization step


  [1]: https://i.sstatic.net/xrVRu.png
  [2]: https://i.sstatic.net/Ts5Xh.png