# Expected value of orthogonal projection $X^{+}X$

Let $$X\in\mathbb{R}^{m\times n}$$, where $$m, be a random matrix where the rows $$x_i$$ ($$i=1,...,m$$) are sampled i.i.d. from Gaussian distribution with mean $$0$$ and covariance $$\Sigma$$, i.e. $$x_i\sim N(0,\Sigma)$$.

How to calculate the expected value $$\mathbb{E}[X^{+}X]$$ where $$X^{+}$$ is the Moore–Penrose inverse of $$X$$ ?

Thank you.

• why for the special case it holds that $\bf E[X^+X]=\frac{m}{n}I$, I am very confused, can you give the reference of this property?
– John
Feb 5, 2023 at 18:31
• It is in fact incorrect for general $m$ and $n$, fixed. Feb 6, 2023 at 9:18
• Can you provide some examples to show that this conclusion is incorrect? Do you mean $$E[X^{+}X]=\frac{m}{n}I$$ does not hold for $m>n$ or this statement has not been proved yet? We test this statement in matlab, and it seems correct when $m\leq n$.
– John
Feb 6, 2023 at 12:14
• Sorry, I was thinking that it is correct only for large $m$ and $n$, but it should be also correct for small values. Just use the SVD of $X$ to show it. Feb 6, 2023 at 16:36

Assume that $$\det\Sigma\ne0$$. Then the random matrix $$X$$ is of rank $$m$$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $$X$$ is $$X^+=X^\top(XX^\top)^{-1}$$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$
It appears that $$EX^+X=EX^\top(XX^\top)^{-1}X$$ cannot be expressed in closed form, even in the fully specified case when $$m=2$$, $$n=3$$, and $$\Sigma=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$$.
Indeed, in this case $$\Sigma=A^\top A$$ for $$A:=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$$. So, for the rows $$x_i$$ of $$X$$ we can write $$x_i=z_i A$$, where the $$z_i$$'s are iid rows of iid standard normal random variables $$z_{i,j}$$.
In the image of a Mathematica notebook below, the expression of even the $$(1,1)$$-entry (P11) of the matrix $$P:=X^+X$$ in terms of the $$z_{i,j}$$'s looks very formidable, and Mathematica cannot do anything for the expectation of P11, leaving it unevaluated after working on it for more than an hour (click on the image to enlarge it):
• Could you explain why $X^{\top}(XX^{\top})^{-1}X=Z^{\top}(ZZ^{\top})^{-1}Z$ ? Thank you. Nov 18, 2021 at 19:28
• Thank you. Do you think it is feasible if $\Sigma$ is diagonal (but not scalar) ? Nov 19, 2021 at 9:49
• @Edward : I have done similarly for the diagonal matrix $\Sigma$ with $1,1,4$ on the diagonal. Then the expression for the orthoprojector is not substantially simpler, and Mathematica still cannot do anything with the expectation. Nov 19, 2021 at 19:47