3
$\begingroup$

For $i= 1, \ldots, n$, let $A_i \in \mathbb{R}^{d \times d}$ be random i.i.d. matrices with $E [A_i] =0$. Can we relate (upper bound) $E[\|\sum_{i=1}^n A_i \|_F^4]$ to $E[\|A_i\|^4_F]$ ?

$\endgroup$
4
$\begingroup$

The $A_i$'s are independent zero-mean random vectors in $\mathbb{R}^{d \times d}$, which is a Hilbert space with respect to the Frobenius norm $\|\cdot\|:=\|\cdot\|_F$. So, by a vector version of Rosenthal's inequality (see e.g. Theorem 5.2), for some real universal constant $K$, \begin{equation} E\Big\|\sum_i A_i\Big\|^4\le K\Big[\sum_iE\|A_i\|^4+\Big(\sum_iE\|A_i\|^2\Big)^2\Big]. \end{equation} In the iid case, we have \begin{align} E\Big\|\sum_i A_i\Big\|^4&\le K[nE\|A_1\|^4+n^2(E\|A_1\|^2)^2] \\ &\le K(n+n^2)E\|A_1\|^4\le 2Kn^2E\|A_1\|^4. \end{align}


More elementarily, denoting by $(xy)$ the inner product $\text{tr}(xy^T)$ of vectors $x,y$ in $\mathbb{R}^{d \times d}$ and letting $[n]:=\{1,\dots,n\}$, we have \begin{equation} E\Big\|\sum_i A_i\Big\|^4=\sum_{(i,j,k,l)\in[n]^4}E(A_iA_j)(A_kA_l). \end{equation} The summand $E(A_iA_j)(A_kA_l)$ is nonzero only if (i) $i=j=k=l$ or (ii) $i=j\ne k=l$ or (iii) $i=k\ne j=l$ or (iv) $i=l\ne j=k$. So, in the iid case, \begin{equation} E\Big\|\sum_i A_i\Big\|^4=nE\|A_1\|^4+n(n-1)(E\|A_1\|^2)^2+2n(n-1)E(A_1A_2)^2, \end{equation} so that, by the Schwarz inequality, \begin{align} nE\|A_1\|^4+n(n-1)(E\|A_1\|^2)^2&\le E\Big\|\sum_i A_i\Big\|^4 \\ &\le nE\|A_1\|^4+3n(n-1)(E\|A_1\|^2)^2 \\ &\le n(3n-2)E\|A_1\|^4. \end{align}

$\endgroup$
  • $\begingroup$ Thanks a lot, Iosif! $\endgroup$ – Kcafe Jan 9 at 20:28

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.