Expected norms of Wishart matrices

Question

Suppose $x_i \stackrel{\text{i.i.d}}{\sim} \mathcal{N}(\mu,\Sigma)$. What can we say about dependence on $b$ of Frobenius/spectral norm quantities below?

$$f(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^T\right\|_F^2$$

$$g(b)=\left\|\frac{1}{b}\sum_{i=1}^b x_i x_i^T\right\|$$

Empirically, the following gives a near perfect fit for some problems, how can this be explained?

$$E\frac{b}{f(b)} \approx \frac{1}{E f(1)}+ \frac{b-1}{Ef(\infty)}$$

$$E\left(\frac{b}{g(b)}\right)^2 \approx \frac{1}{E g(1)}+ \frac{b-1}{Eg(\infty)}$$

For instance, here's a check of this formula against empirically estimated quantities for 1000-dimensional Gaussian centered at zero and covariance matrix having eigenvalues $1,\frac{1}{2},\ldots,\frac{1}{1000}$.

notebook

Any pointers appreciated!

Edit we can write $f(b)$ in terms of Wishart random variable $W_b$ ($b$ degrees of freedom, covariance $\Sigma$). The following quantity has a simple dependence on $b$ (first graph)

$$E\left[\frac{b}{f(b)}\right]=E\left[\frac{b^3}{\operatorname{Tr}(W_b W_b^T)}\right]$$

Motivation: expected value of dot product squared for a random pair of vectors in a batch of size $b$ converges to $\operatorname{Tr}E[xx']^2$, behavior above suggests a way to estimate this value for finite $b$ which in term informs the largest useful batch size in mini-batch SGD, related question

Answer 1

Answering Frobenius norm part of the question, $f(b)$. Still curious how to do the equivalent for $g(b)$

---

Suppose $X$ contains $b$ IID instances of random variable $x$ stacked as rows. Let $x$ be distributed as zero-centered Gaussian with covariance $\Sigma$. We can show the following

$$f(b)=\frac{1}{b^2}E[\|X'X\|_F^2]=\frac{(b+1)}{b}\operatorname{Tr}\Sigma^2+\frac{1}{b} (\operatorname{Tr} \Sigma)^2$$

To prove this, first note that:

$$\|X^T X\|_F^2=\operatorname{Tr} X^T X X^T X$$

And that for arbitrary R.V. $x$ we have $$E[X'XX'X]=bE[xx'xx']+b(b-1)E[xx']E[xx']$$

For Gaussian $x$ centered at zero we can apply Wick's theorem to get:

$$E[xx'xx']=2E[xx']E[xx']+E[xx'] \operatorname{Tr}E[xx']$$

Combining these three equations yields the result above