Powers of Frobenius norm of sum of random matrices

Question

For $i= 1, \ldots, n$, let $A_i \in \mathbb{R}^{d \times d}$ be random i.i.d. matrices with $E [A_i] =0$. Can we relate (upper bound) $E[\|\sum_{i=1}^n A_i \|_F^4]$ to $E[\|A_i\|^4_F]$ ?

Answer 1

The $A_i$'s are independent zero-mean random vectors in $\mathbb{R}^{d \times d}$, which is a Hilbert space with respect to the Frobenius norm $\|\cdot\|:=\|\cdot\|_F$. So, by a vector version of Rosenthal's inequality (see e.g. Theorem 5.2), for some real universal constant $K$, \begin{equation} E\Big\|\sum_i A_i\Big\|^4\le K\Big[\sum_iE\|A_i\|^4+\Big(\sum_iE\|A_i\|^2\Big)^2\Big]. \end{equation} In the iid case, we have \begin{align} E\Big\|\sum_i A_i\Big\|^4&\le K[nE\|A_1\|^4+n^2(E\|A_1\|^2)^2] \\ &\le K(n+n^2)E\|A_1\|^4\le 2Kn^2E\|A_1\|^4. \end{align}

---

More elementarily, denoting by $(xy)$ the inner product $\text{tr}(xy^T)$ of vectors $x,y$ in $\mathbb{R}^{d \times d}$ and letting $[n]:=\{1,\dots,n\}$, we have \begin{equation} E\Big\|\sum_i A_i\Big\|^4=\sum_{(i,j,k,l)\in[n]^4}E(A_iA_j)(A_kA_l). \end{equation} The summand $E(A_iA_j)(A_kA_l)$ is nonzero only if (i) $i=j=k=l$ or (ii) $i=j\ne k=l$ or (iii) $i=k\ne j=l$ or (iv) $i=l\ne j=k$. So, in the iid case, \begin{equation} E\Big\|\sum_i A_i\Big\|^4=nE\|A_1\|^4+n(n-1)(E\|A_1\|^2)^2+2n(n-1)E(A_1A_2)^2, \end{equation} so that, by the Schwarz inequality, \begin{align} nE\|A_1\|^4+n(n-1)(E\|A_1\|^2)^2&\le E\Big\|\sum_i A_i\Big\|^4 \\ &\le nE\|A_1\|^4+3n(n-1)(E\|A_1\|^2)^2 \\ &\le n(3n-2)E\|A_1\|^4. \end{align}