Skip to main content
3 of 3
added 768 characters in body
Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229

The $A_i$'s are independent zero-mean random vectors in $\mathbb{R}^{d \times d}$, which is a Hilbert space with respect to the Frobenius norm $\|\cdot\|:=\|\cdot\|_F$. So, by a vector version of Rosenthal's inequality (see e.g. Theorem 5.2), for some real universal constant $K$, \begin{equation} E\Big\|\sum_i A_i\Big\|^4\le K\Big[\sum_iE\|A_i\|^4+\Big(\sum_iE\|A_i\|^2\Big)^2\Big]. \end{equation} In the iid case, we have \begin{align} E\Big\|\sum_i A_i\Big\|^4&\le K[nE\|A_1\|^4+n^2(E\|A_1\|^2)^2] \\ &\le K(n+n^2)E\|A_1\|^4\le 2Kn^2E\|A_1\|^4. \end{align}


More elementarily, denoting by $(xy)$ the inner product $\text{tr}(xy^T)$ of vectors $x,y$ in $\mathbb{R}^{d \times d}$ and letting $[n]:=\{1,\dots,n\}$, we have \begin{equation} E\Big\|\sum_i A_i\Big\|^4=\sum_{(i,j,k,l)\in[n]^4}E(A_iA_j)(A_kA_l). \end{equation} The summand $E(A_iA_j)(A_kA_l)$ is nonzero only if (i) $i=j=k=l$ or (ii) $i=j\ne k=l$ or (iii) $i=k\ne j=l$ or (iv) $i=l\ne j=k$. So, in the iid case, \begin{equation} E\Big\|\sum_i A_i\Big\|^4=nE\|A_1\|^4+n(n-1)(E\|A_1\|^2)^2+2n(n-1)E(A_1A_2)^2, \end{equation} so that, by the Schwarz inequality, \begin{align} nE\|A_1\|^4+n(n-1)(E\|A_1\|^2)^2&\le E\Big\|\sum_i A_i\Big\|^4 \\ &\le nE\|A_1\|^4+3n(n-1)(E\|A_1\|^2)^2 \\ &\le n(3n-2)E\|A_1\|^4. \end{align}

Iosif Pinelis
  • 127.8k
  • 8
  • 107
  • 229