Return to Answer

Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:

$G=\sum_j \sum_{l\ne j} w_j^Tw_l w_j^TCw_l = \sum_{j\ne l, ik} w_{ji}w_{li} c_i w_{jk} w_{lk}$. Write this quantity as $$ \begin{split} G=\sum_{j\ne l, i\ne k} w_{ji}w_{li} c_i w_{jk} w_{lk} &+ \sum_{j\ne l, i=k} (w_{ij}^2-1)(w_{lj}^2 -1)c_i \\ &+(w_{ij}^2-1)c_i + (w_{lj}^2 -1)c_i +c_i \end{split} $$ This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is $$ E[(G-m(m-1)trace[C])^2]=\sum_{j\ne l, i \ne k} c_i^2 + \sum_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c_i^2. $$ $$= m(m-1)\|C\|_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$ The dominant term is of order $m^2d \|C\|_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 >> Var[G]$, that is, $$ m^2 trace[C] >> \|C\|_F m \sqrt{d}. $$

For the diagonal terms, we have $\sum_j d w_j^TCw_j + \sum_{j} (\|w_j\|^2-d)w_j^TCw_j$. The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|_F^2$. Again, the mean dominates the standard deviation if and only if $$ m d ~trace[C] \gg \sqrt m d \|C\|_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.

Edit: since $\|C\|_F^2 \le trace[C]^2$ for $C$ psd, these conditions are always satisfied.

Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:

$G=\sum_j \sum_{l\ne j} w_j^Tw_l w_j^TCw_l = \sum_{j\ne l, ik} w_{ji}w_{li} c_i w_{jk} w_{lk}$. Write this quantity as $$ \begin{split} G=\sum_{j\ne l, i\ne k} w_{ji}w_{li} c_i w_{jk} w_{lk} &+ \sum_{j\ne l, i=k} (w_{ij}^2-1)(w_{lj}^2 -1)c_i \\ &+(w_{ij}^2-1)c_i + (w_{lj}^2 -1)c_i +c_i \end{split} $$ This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is $$ E[(G-m(m-1)trace[C])^2]=\sum_{j\ne l, i \ne k} c_i^2 + \sum_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c_i^2. $$ $$= m(m-1)\|C\|_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$ The dominant term is of order $m^2d \|C\|_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 \gg Var[G]$, that is, $$ m^2 trace[C] \gg \|C\|_F m \sqrt{d}. $$

For the diagonal terms, we have $\sum_j d w_j^TCw_j + \sum_{j} (\|w_j\|^2-d)w_j^TCw_j$. The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|_F^2$. Again, the mean dominates the standard deviation if and only if $$ m d ~trace[C] \gg \sqrt m d \|C\|_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.

Edit: since $\|C\|_F^2 \le trace[C]^2$ for $C$ psd, these conditions are always satisfied.

Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:

$G=\sum_j \sum_{l\ne j} w_j^Tw_l w_j^TCw_l = \sum_{j\ne l, ik} w_{ji}w_{li} c_i w_{jk} w_{lk}$. Write this quantity as $$ \begin{split} G=\sum_{j\ne l, i\ne k} w_{ji}w_{li} c_i w_{jk} w_{lk} &+ \sum_{j\ne l, i=k} (w_{ij}^2-1)(w_{lj}^2 -1)c_i \\ &+(w_{ij}^2-1)c_i + (w_{lj}^2 -1)c_i +c_i \end{split} $$ This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is $$ E[(G-m(m-1)trace[C])^2]=\sum_{j\ne l, i \ne k} c_i^2 + \sum_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c_i^2. $$ $$= m(m-1)\|C\|_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$ The dominant term is of order $m^2d \|C\|_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 >> Var[G]$, that is, $$ m^2 trace[C] >> \|C\|_F m \sqrt{d}. $$

For the diagonal terms, we have $\sum_j d w_j^TCw_j + \sum_{j} (\|w_j\|^2-d)w_j^TCw_j$. The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|_F^2$. Again, the mean dominates the standard deviation if and only if $$ m d ~trace[C] \gg \sqrt m d \|C\|_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.

Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:

$G=\sum_j \sum_{l\ne j} w_j^Tw_l w_j^TCw_l = \sum_{j\ne l, ik} w_{ji}w_{li} c_i w_{jk} w_{lk}$. Write this quantity as $$ \begin{split} G=\sum_{j\ne l, i\ne k} w_{ji}w_{li} c_i w_{jk} w_{lk} &+ \sum_{j\ne l, i=k} (w_{ij}^2-1)(w_{lj}^2 -1)c_i \\ &+(w_{ij}^2-1)c_i + (w_{lj}^2 -1)c_i +c_i \end{split} $$ This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is $$ E[(G-m(m-1)trace[C])^2]=\sum_{j\ne l, i \ne k} c_i^2 + \sum_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c_i^2. $$ $$= m(m-1)\|C\|_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$ The dominant term is of order $m^2d \|C\|_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 >> Var[G]$, that is, $$ m^2 trace[C] >> \|C\|_F m \sqrt{d}. $$

For the diagonal terms, we have $\sum_j d w_j^TCw_j + \sum_{j} (\|w_j\|^2-d)w_j^TCw_j$. The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|_F^2$. Again, the mean dominates the standard deviation if and only if $$ m d ~trace[C] \gg \sqrt m d \|C\|_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.

Edit: since $\|C\|_F^2 \le trace[C]^2$ for $C$ psd, these conditions are always satisfied.

Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:

$G=\sum_j \sum_{l\ne j} w_j^Tw_l w_j^TCw_l = \sum_{j\ne l, ik} w_{ji}w_{li} c_i w_{jk} w_{lk}$. Write this quantity as $$ G=\sum_{j\ne l, i\ne k} w_{ji}w_{li} c_i w_{jk} w_{lk} + \sum_{j\ne l, i=k} (w_{ij}^2-1)(w_{lj}^2 -1)c_i + (w_{ij}^2-1)c_i + (w_{lj}^2 -1)c_i +c_i $$ This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is $$ E[(G-m(m-1)trace[C])^2]=\sum_{j\ne l, i \ne k} c_i^2 + \sum_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c_i^2. $$ $$= m(m-1)\|C\|_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$ The dominant term is of order $m^2d \|C\|_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 >> Var[G]$, that is, $$ m^2 trace[C] >> \|C\|_F m \sqrt{d}. $$

For the diagonal terms, we have $\sum_j d w_j^TCw_j + \sum_{j} (\|w_j\|^2-d)w_j^TCw_j$. The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|_F^2$. Again, the mean dominates the standard deviation if and only if $$ m d ~trace[C] >> \sqrt m d \|C\|_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.

Assume iid $N(0,1)$ entries, assume $C$ diagonal, and focus first on the non-diagonal terms:

$G=\sum_j \sum_{l\ne j} w_j^Tw_l w_j^TCw_l = \sum_{j\ne l, ik} w_{ji}w_{li} c_i w_{jk} w_{lk}$. Write this quantity as $$ \begin{split} G=\sum_{j\ne l, i\ne k} w_{ji}w_{li} c_i w_{jk} w_{lk} &+ \sum_{j\ne l, i=k} (w_{ij}^2-1)(w_{lj}^2 -1)c_i \\ &+(w_{ij}^2-1)c_i + (w_{lj}^2 -1)c_i +c_i \end{split} $$ This is a decomposition in uncorrelated polynomials (any two terms are uncorrelated), so that the second moment is $$ E[(G-m(m-1)trace[C])^2]=\sum_{j\ne l, i \ne k} c_i^2 + \sum_{j\ne l, i}(E[(Z^2-1)^2]^2 + 2 E[(Z^2-1)^2])c_i^2. $$ $$= m(m-1)\|C\|_F^2((d-1)+E[(Z^2-1)^2]^2 + 2E[(Z^2-1)^2]).$$ The dominant term is of order $m^2d \|C\|_F^2$, while the mean is $m(m-1)trace[C]$. Hence $G/E[G]-1$ converges to 0 in probability (or in L2) provided that $E[G]^2 >> Var[G]$, that is, $$ m^2 trace[C] >> \|C\|_F m \sqrt{d}. $$

For the diagonal terms, we have $\sum_j d w_j^TCw_j + \sum_{j} (\|w_j\|^2-d)w_j^TCw_j$. The second term is negligible compared to the first one if you use $\chi^2$ concentration (e..g, Bernstein inequality) for $\|w_j\|^2-d$, while the first term has mean $md trace[C]$ and variance $2md^2\|C\|_F^2$. Again, the mean dominates the standard deviation if and only if $$ m d ~trace[C] \gg \sqrt m d \|C\|_F.$$ This is equivalent to the condition on the non-diagonal terms if $m\asymp d$.