6
$\begingroup$

Let $X\in\mathbb{R}^{m\times n}$, where $m<n$, be a random matrix where the rows $x_i$ ($i=1,...,m$) are sampled i.i.d. from Gaussian distribution with mean $0$ and covariance $\Sigma$, i.e. $x_i\sim N(0,\Sigma)$.

How to calculate the expected value $\mathbb{E}[X^{+}X]$ where $X^{+}$ is the Moore–Penrose inverse of $X$ ?

Thank you.

$\endgroup$
4
  • $\begingroup$ why for the special case it holds that $\bf E[X^+X]=\frac{m}{n}I$, I am very confused, can you give the reference of this property? $\endgroup$
    – John
    Feb 5, 2023 at 18:31
  • $\begingroup$ It is in fact incorrect for general $m$ and $n$, fixed. $\endgroup$
    – Edward
    Feb 6, 2023 at 9:18
  • $\begingroup$ Can you provide some examples to show that this conclusion is incorrect? Do you mean $$E[X^{+}X]=\frac{m}{n}I$$ does not hold for $m>n$ or this statement has not been proved yet? We test this statement in matlab, and it seems correct when $m\leq n$. $\endgroup$
    – John
    Feb 6, 2023 at 12:14
  • 1
    $\begingroup$ Sorry, I was thinking that it is correct only for large $m$ and $n$, but it should be also correct for small values. Just use the SVD of $X$ to show it. $\endgroup$
    – Edward
    Feb 6, 2023 at 16:36

1 Answer 1

3
$\begingroup$

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$

It appears that $EX^+X=EX^\top(XX^\top)^{-1}X$ cannot be expressed in closed form, even in the fully specified case when $m=2$, $n=3$, and $\Sigma=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$.

Indeed, in this case $\Sigma=A^\top A$ for $A:=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$. So, for the rows $x_i$ of $X$ we can write $x_i=z_i A$, where the $z_i$'s are iid rows of iid standard normal random variables $z_{i,j}$.

In the image of a Mathematica notebook below, the expression of even the $(1,1)$-entry (P11) of the matrix $P:=X^+X$ in terms of the $z_{i,j}$'s looks very formidable, and Mathematica cannot do anything for the expectation of P11, leaving it unevaluated after working on it for more than an hour (click on the image to enlarge it):

enter image description here

$\endgroup$
4
  • 1
    $\begingroup$ Could you explain why $X^{\top}(XX^{\top})^{-1}X=Z^{\top}(ZZ^{\top})^{-1}Z$ ? Thank you. $\endgroup$
    – Edward
    Nov 18, 2021 at 19:28
  • $\begingroup$ @Edward : That equality was incorrect, and the answer has been changed accordingly. $\endgroup$ Nov 19, 2021 at 0:45
  • $\begingroup$ Thank you. Do you think it is feasible if $\Sigma$ is diagonal (but not scalar) ? $\endgroup$
    – Edward
    Nov 19, 2021 at 9:49
  • $\begingroup$ @Edward : I have done similarly for the diagonal matrix $\Sigma$ with $1,1,4$ on the diagonal. Then the expression for the orthoprojector is not substantially simpler, and Mathematica still cannot do anything with the expectation. $\endgroup$ Nov 19, 2021 at 19:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.