Does $E[1/f]\overset{d}\to 1/E[f]$ for $\operatorname{Tr}H=1,\operatorname{Tr}H^2=0.5$? Suppose $x$ is a Gaussian random variable in $d$ dimensions with $H=E[xx^T],\ \operatorname{Tr}(H)=1,\operatorname{Tr}(H^2)=0.5$. Take $m$ I.I.D. samples of $x$ and stack them as rows of $X$.
Is it possible to show that the following holds as $d\to \infty$?
$$E\left[\frac{1}{\operatorname{Tr}X^T X X^T X}\right] \to \frac{1}{E\operatorname{Tr}(X^T X X^T X]}$$
If we assume that spectrum of $H$ obeys power law decay, we can take $\lambda_i = c_1 i^{c_2}$, and solve for $c1,c2$ to satisfy the constraints. Taking $d$ to infinity, the gap in approximation shrinks.

*

*Does the gap shrink to zero?

*Is this behavior specific to power law decay?


If this holds for slightly more general $\operatorname{Tr}X^T X A X^T X$, we can choose $A=I$ to prove this and $A=\mathbb{1}\mathbb{1}^T$ to prove this
 A: $\newcommand{\tr}{\operatorname{tr}}\newcommand{\de}{\delta} $
Let us show that the desired convergence will never take place.
First here, note that
\begin{equation*}
U_d:=\tr(X^TXX^TX)=\sum_{a,b=1}^d\sum_{j,k=1}^m x_{ja}^2x_{kb}^2,  \tag{1}\label{1} 
\end{equation*}
where $x_{ja}$ is the $a$th coordinate of the random vector $x_j$, given the iid copies $x_1,\dots,x_m$ of the random vector $x$, with $Exx^T=H$, $\tr H=1$, and $\tr(H^2)=1/2$. Since no conditions on $Ex$ are specified, assume for simplicity that $Ex=0$. Then the conditions $Exx^T=H$, $\tr H=1$, and $\tr(H^2)=1/2$ imply
\begin{equation*}
    EU_d=m(m+1). \tag{1.5}\label{1.5}
\end{equation*}
So, the desired convergence is
\begin{equation*}
    E\frac1{U_d}\to\frac1{EU_d}=\frac1{m(m+1)} \tag{2}\label{2} 
\end{equation*}
as $d\to\infty$.
This would imply that
\begin{equation*}
    V_d:=\dfrac{U_d}{EU_d}\to1 \tag{2.5}\label{2.5}
\end{equation*}
(in probability, as $d\to\infty$), because for each $\de\in(0,1)$
\begin{equation*}
    \frac{\de^2}{1+\de}\,E1(|V_d-1|\ge\de)\le E\frac{(V_d-1)^2}{V_d}=EV_d+E\frac1{V_d}-2\to0. 
\end{equation*}
Let us diagonalize the matrix $H=Exx^T$ by writing $H=QDQ^T$ for a diagonal matrix $D$ and an orthogonal matrix $Q$. Letting then $y:=Q^T x$ and $y_j:=Q^T x_j$, we have $X^T X=Q_m Y^T Y Q_m^T$, where $Q_m$ is the block-diagonal matrix with $m$ diagonal blocks each equal $Q$ and $Y:=[y_1,\dots,y_m]^T$. So, $U_d=\tr(X^TXX^TX)=\tr(Y^TYY^TY)$. Thus, wlog we have \eqref{1} with all the random variables (r.v.'s) $x_{ja}$ with $(j,a)\in[m]\times[d]$ independent (and centered Gaussian); as usual, $[m]:=\{1,\dots,m\}$. Also, the distribution of $x_{ja}$ does not depend on $j$.
Now straightforward (but long) calculations show that
\begin{equation*}
\begin{aligned}
    EU_d^2&=m (m+2) (m+4) (m+6) t_8 \\ 
    &+4 m^2 (m+2) (m+4)(t_6 t_2-t_8) \\ 
    &+3 m^2 (m+2)^2(t_4^2-t_8) \\ 
    &+6 m^3 (m+2)(t_4 t_2^2-2 t_6 t_2+2t_8-t_4^2) \\ 
    &+\frac{24}{4} m^4\Big(\frac{t_2^4}{6}-t_2^2 t_4+\frac{4t_2t_6}{3}+\frac{t_4^2}{2}-t_8\Big), 
\end{aligned}
\tag{3}\label{3}    
\end{equation*}
where $t_{2p}:=\tr(D^p)=\tr(H^p)$, so that $t_2=1$ and $t_4=1/2$, and hence
\begin{equation*}
\begin{aligned}
    EU_d^2&=m (m^3 +6 m^2 +  m (3 + 32 t_6) + 48 t_8).  
\end{aligned}
\end{equation*}
The minimum of the latter expression given the logconvexity conditions $t_6\ge t_4^2/t_2=1/4$ and $t_8\ge t_6^2/t_4=2t_6^2$ is $m^4+6 m^3+11 m^2+6 m=(m(m+1))^2+2 m (m+1) (2 m+3)$ and hence, in view of \eqref{1.5},
\begin{equation*}
    Var\,V_d=\frac{EU_d^2}{(EU_d)^2}-1\ge\frac{2 (2 m+3)}{m (m+1)}. \tag{4}\label{4}    
\end{equation*}
It is similar to but much easier than \eqref{3} to show that $EU_d^4=O(1)$ and hence $EV_d^4=O(1)$, in view of \eqref{1.5} (where the constants in $O(\cdot)$ can depend only on $m$). So, $(V_d-EV_d)^2=(V_d-1)^2$ is uniformly integrable in $d$. So, in view of \eqref{2.5}, $Var\,V_d=E(V_d-1)^2\to0$ as $d\to\infty$, which contradicts \eqref{4}. $\quad\Box$
