Skip to main content
added 616 characters in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting

It appears that $Z$$EX^+X=EX^\top(XX^\top)^{-1}X$ cannot be expressed in closed form, even in the matrix with rowsfully specified case when $z_i:=\Sigma^{-1/2}x_i$$m=2$, $n=3$, and $\Sigma=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$.

Indeed, in this case $\Sigma=A^\top A$ for $A:=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$. So, we see thatfor the rows $x_i$ of $X$ we can write $x_i=z_i A$, where the $z_i$'s are iid rows of iid standard normal random variables $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So$z_{i,j}$.

In the image of a Mathematica notebook below, the problem reduces toexpression of even the case when $\Sigma=I_n$ -$(1,1)$- whichentry (P11) of the matrix $P:=X^+X$ in terms of the $z_{i,j}$'s looks very formidable, as you notedand Mathematica cannot do anything for the expectation of P11, is easy.leaving it unevaluated after working on it for more than an hour (click on the image to enlarge it):

Thus, the answer is $$EX^+X=\frac mn\,I_n,$$ as long as $\det\Sigma\ne0$.enter image description here

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$ -- which, as you noted, is easy.

Thus, the answer is $$EX^+X=\frac mn\,I_n,$$ as long as $\det\Sigma\ne0$.

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$

It appears that $EX^+X=EX^\top(XX^\top)^{-1}X$ cannot be expressed in closed form, even in the fully specified case when $m=2$, $n=3$, and $\Sigma=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$.

Indeed, in this case $\Sigma=A^\top A$ for $A:=\left( \begin{array}{ccc} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{array} \right)$. So, for the rows $x_i$ of $X$ we can write $x_i=z_i A$, where the $z_i$'s are iid rows of iid standard normal random variables $z_{i,j}$.

In the image of a Mathematica notebook below, the expression of even the $(1,1)$-entry (P11) of the matrix $P:=X^+X$ in terms of the $z_{i,j}$'s looks very formidable, and Mathematica cannot do anything for the expectation of P11, leaving it unevaluated after working on it for more than an hour (click on the image to enlarge it):

enter image description here

added 8 characters in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$, -- which, as you saidnoted, is easy.

Thus, the answer is $$EX^+X=\frac mn\,I_n,$$ as long as $\det\Sigma\ne0$.

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$, which you said is easy.

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$ -- which, as you noted, is easy.

Thus, the answer is $$EX^+X=\frac mn\,I_n,$$ as long as $\det\Sigma\ne0$.

added 1 character in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$$$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$, which you said is easy.

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$, which you said is easy.

Assume that $\det\Sigma\ne0$. Then the random matrix $X$ is of rank $m$ almost surely (a.s.). So, a.s. the Moore--Penrose inverse of $X$ is $X^+=X^\top(XX^\top)^{-1}$ and hence $$X^+X=X^\top(XX^\top)^{-1}X.$$ Letting $Z$ be the matrix with rows $z_i:=\Sigma^{-1/2}x_i$, we see that the $z_i$'s are iid $N(0,I_n)$, and $$X^+X=X^\top(XX^\top)^{-1}X=Z^\top(ZZ^\top)^{-1}Z=Z^+Z.$$ So, the problem reduces to the case when $\Sigma=I_n$, which you said is easy.

deleted 1 character in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229
Loading
deleted 1 character in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229
Loading
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229
Loading