Let $F=F_{m,d}$ be a random $m \times d$ matrix with iid entries from $N(0,1)$. Let $A=A_d$ and $B=B_d$ be deterministic $d \times d$ positive-definite matrices. In case it helps, it may be assumed that
- $A$ and $B$ are diagonal.
- $A$ and $B$ are diagonal with eigenvalues given by $a_i \asymp i^{-\beta_1}$ and $b_i \asymp i^{-\beta_2}$, for some constants $\beta_1,\beta_2 \gt 1$.
- $A$, $B$, or (inclusive) $AB^{-1}$ has limiting spectral density (LSD) in the limit $d \to \infty$.
Question 1. What is the limiting value of $\dfrac{1}{m}\mathrm{tr}(FAF^\top (FBF^\top)^{-1})$ in the limit $m,d \to \infty$ such that $d/m \to \gamma \in (1,\infty)$ ?
Question 2. Same as Question 1, but with $F$ uniform over the orthogonal group $FF^\top = I_m$.
Note that if $A=B$, then the sought-for limit is $1$.
Observation
If $A$ and $B$ are aligned in the sense that they have the same eigenvectors, with eigenvalues $a_1 \ge a_2 \ge \ldots \ge a_d$ and $b_1 \ge b_2 \ge \ldots \ge b_d$ respectively, and $F$ is as in Question 2, then it is easy to see that $$ \dfrac{1}{m}\mathrm{tr}(FAF^\top (FBF^\top)^{-1}) \overset{D}{=} \frac{1}{m} \sum_{i=1}^m \frac{a_{\pi(i)}}{b_{\pi(i)}}, $$
where $\pi$ is a uniformly random permutation of $\{1,2,\ldots,d\}$.
Furthermore, it's not hard to then deduce that $$ \mathbb E\, \left[\dfrac{1}{m}\mathrm{tr}(FAF^\top (FBF^\top)^{-1})\right] = \frac{1}{m}\sum_{i=1}^m \frac{a_i}{b_i}. $$
