Let $X \in \mathbb R^{n \times d}$ be a random matrix with iid rows from $N(0,\Sigma_d)$ where $\Sigma_d$ is a $d \times d$ psd matrix verifying w.h.p,
- $\mbox{trace}(\Sigma_d/d)= 1$.
- $\|\Sigma_d\|_{op},\|\Sigma_d^{-1}\|_{op} = \mathcal O(1)$.
Define $R := Q\circ \overline{Q}$, the Hadamard (i.e element-wise) product of $Q:=XX^\top/d$ and $\overline{Q} := X\Sigma_dX^\top/d$.
Question. Does there exist absolute constants $a,b \in \mathbb R$ such that w.h.p $\|R-R_0\|_{op} \lesssim n^{-1/2}\log^c n$, where $R_0 := a Q + b I_n$ ?
Solution to isotropic case
If $\Sigma_d = I_d$., then $\overline{Q} = Q$, and so $R = Q \circ Q/d=f(Q):=(f(q_{ij}/d)_{i,j \in [n]}$, where $h:\mathbb R \to \mathbb R$ is defined by $f(t) = t^2$. By, Theorem 2.3 of El Karoui (2010), we know that we may take $$ \begin{split} d \cdot R_0 &= \left(f(0)+\dfrac{f''(0)\mbox{trace}(\Sigma_d^2))}{2d^2}\right)1_n1_n^\top + f'(0)Q+(f(1)-f(0)-f'(0))I_n\\ & = (I_n+\Delta), \end{split} $$ where $\|\Delta\|_{op} \lesssim n^{-1/2}\log^c n$ w.h.p, so that $\|R-d^{-1}I_n\|_{op} \lesssim d^{-1/2}\log^c n$ w.h.p. We deduce that $\|R\|_{op} \lesssim d^{-1/2}\log^c n$ w.h.p.
