How to analyze the value of convergence of functions of random matrices?

Question

Consider a random i.i.d matrix $\mathbf{A}_{m\times n}$ with entries generated from a complex Gaussian distribution with zero mean and unit variance. I am interested in the large dimension analysis of the expression ($m,n\rightarrow\infty$ with fixed $m/n$): \begin{align} \label{eq:lll} \mathbf{a}^\mathrm{H}\left(\frac{1}{n}\mathbf{A}\mathbf{A}^\mathrm{H}+\mathbf{I}_m\right)^{-1}\mathbf{a}, \end{align} where $\mathbf{I}_m$ represents the identity matrix and $\mathbf{a}$ is one of the columns of $\mathbf{A}$. According to the law of large numbers, we know that $\frac{1}{n}\mathbf{A}\mathbf{A}^\mathrm{H}\rightarrow\mathbf{I}_m$. I expect the following result to hold: \begin{align} \left(\frac{1}{n}\mathbf{A}\mathbf{A}^\mathrm{H}+\mathbf{I}_m\right)^{-1}\rightarrow 0.5\mathbf{I}_m. \end{align} However, my Matlab simulation does not verify this. I have the following questions:

1- Can we analytically compute the convergence value of $\mathbf{a}^\mathrm{H}\left(\frac{1}{n}\mathbf{A}\mathbf{A}^\mathrm{H}+\mathbf{I}_m\right)^{-1}\mathbf{a}$?

2- Are there any sources that provide clear explanations about the convergence of functions of i.i.d matrices?

Answer 1

Assume real entries. Let $b$ be a column of $A$ and write $\tilde A$ the matrix $A$ with that column removed so that $AA^T = \tilde A\tilde A^T + bb^T$. By the https://en.wikipedia.org/wiki/Sherman%E2%80%93Morrison_formula, $$ b^T(AA^T + n I_m)^{-1}b = b^T(\tilde A\tilde A^T + nI_m)^{-1}b - \frac{b^T(\tilde A\tilde A^T + nI_m)^{-1}bb^T(\tilde A\tilde A^T + nI_m)^{-1}b}{1 + b^T(\tilde A\tilde A^T/n + I_m)^{-1}b}. $$ This can be written $Z-Z^2/(1+Z) = Z/(1+Z)$ for $Z=b^T(\tilde A\tilde A^T + n I_m)^{-1}b$. Now by independence, since $b$ is normal, by the Hanson-Wright inequality (concentration of quadratic forms), $Z \approx trace[(\tilde A\tilde A^T + n I_m)^{-1}]$. The quantity $trace[(\tilde A\tilde A^T + n I_m)^{-1}]$ can be characterized by the limiting spectral distribution of $\tilde A\tilde A^T/n$ given by https://en.wikipedia.org/wiki/Marchenko%E2%80%93Pastur_distribution.

Answer 2

We assume that $A$ is a real matrix. Let $B=[b_{i,j}]=\dfrac{1}{n}AA^T$.

$b_{i,i}=1/n ||U||^2$, where $U=[u_j]$ is the $i^{th}$ row of $A$.

$E(b_{i,i})=1/n\sum_j E({u_j}^2)=1$ and $var(b_{i,i})=\dfrac{1}{n^2}n.var({u_j}^2)=\dfrac{2}{n}$;

In the same way, if $i\not= j$, then $E(b_{i,j})=0$ and $var(b_{i,j})=\dfrac{1}{n}$.

With probability $997/1000$, $|X-E(X)|\leq 3\sigma(X)$ (for a normal law).

Then, "in general", $|b_{i,j}-\delta_{i,j}|\leq \dfrac{3\sqrt{2}}{\sqrt{n}}=O(1/\sqrt{n})$.

Finally, when $n$ is great, the $b_{i,j}$'s are close to the $\delta_{i,j}$'s and $a^T(B+I_m)^{-1}a$ is -entrywise- close to $\dfrac{1}{2}||a||^2$.