Skip to main content
added 134 characters in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

$\newcommand\R{\Bbb R}\newcommand\Si{\Sigma}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution (see the details on this below), it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed (over $G_{n,d}$ if $\Si$ is nonsingular, as it is usually assumed, or, more generally, over $G_{n,r}$ where $r$ is the rank of $\Si$). $\quad\Box$


Details on why $QX$ equals $X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$: Write $X=[X_{ij}]_{i\in[n],j\in[d]}$ and $Q=[Q_{ij}]_{i\in[n],j\in[n]}$, where $[n]:=\{1,\dots,n\}$. Then the entries $(QX)_{ij}=\sum_{k\in[n]}Q_{ik}X_{k,j}$ of the matrix $QX$ are zero-mean jointly normal random variables, and for all $i,i'$ in $[n]$ and $j,j'$ in $[d]$ we have $$E(QX)_{ij}(QX)_{i'j'} =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}EX_{kj}X_{k'j'} \\ =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}1(k=k')\Si_{jj'} =\sum_{k\in[n]}Q_{ik}Q_{i'k}\Si_{jj'} =1(i=i')\Si_{jj'},$$ so that the covariances of the $(QX)_{ij}$'s do not depend on $Q$.

$\newcommand\R{\Bbb R}\newcommand\Si{\Sigma}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution (see the details on this below), it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed. $\quad\Box$


Details on why $QX$ equals $X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$: Write $X=[X_{ij}]_{i\in[n],j\in[d]}$ and $Q=[Q_{ij}]_{i\in[n],j\in[n]}$, where $[n]:=\{1,\dots,n\}$. Then the entries $(QX)_{ij}=\sum_{k\in[n]}Q_{ik}X_{k,j}$ of the matrix $QX$ are zero-mean jointly normal random variables, and for all $i,i'$ in $[n]$ and $j,j'$ in $[d]$ we have $$E(QX)_{ij}(QX)_{i'j'} =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}EX_{kj}X_{k'j'} \\ =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}1(k=k')\Si_{jj'} =\sum_{k\in[n]}Q_{ik}Q_{i'k}\Si_{jj'} =1(i=i')\Si_{jj'},$$ so that the covariances of the $(QX)_{ij}$'s do not depend on $Q$.

$\newcommand\R{\Bbb R}\newcommand\Si{\Sigma}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution (see the details on this below), it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed (over $G_{n,d}$ if $\Si$ is nonsingular, as it is usually assumed, or, more generally, over $G_{n,r}$ where $r$ is the rank of $\Si$). $\quad\Box$


Details on why $QX$ equals $X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$: Write $X=[X_{ij}]_{i\in[n],j\in[d]}$ and $Q=[Q_{ij}]_{i\in[n],j\in[n]}$, where $[n]:=\{1,\dots,n\}$. Then the entries $(QX)_{ij}=\sum_{k\in[n]}Q_{ik}X_{k,j}$ of the matrix $QX$ are zero-mean jointly normal random variables, and for all $i,i'$ in $[n]$ and $j,j'$ in $[d]$ we have $$E(QX)_{ij}(QX)_{i'j'} =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}EX_{kj}X_{k'j'} \\ =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}1(k=k')\Si_{jj'} =\sum_{k\in[n]}Q_{ik}Q_{i'k}\Si_{jj'} =1(i=i')\Si_{jj'},$$ so that the covariances of the $(QX)_{ij}$'s do not depend on $Q$.

added 698 characters in body
Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

$\newcommand\R{\Bbb R}$$\newcommand\R{\Bbb R}\newcommand\Si{\Sigma}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution (see the details on this below), it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed. $\quad\Box$


Details on why $QX$ equals $X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$: Write $X=[X_{ij}]_{i\in[n],j\in[d]}$ and $Q=[Q_{ij}]_{i\in[n],j\in[n]}$, where $[n]:=\{1,\dots,n\}$. Then the entries $(QX)_{ij}=\sum_{k\in[n]}Q_{ik}X_{k,j}$ of the matrix $QX$ are zero-mean jointly normal random variables, and for all $i,i'$ in $[n]$ and $j,j'$ in $[d]$ we have $$E(QX)_{ij}(QX)_{i'j'} =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}EX_{kj}X_{k'j'} \\ =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}1(k=k')\Si_{jj'} =\sum_{k\in[n]}Q_{ik}Q_{i'k}\Si_{jj'} =1(i=i')\Si_{jj'},$$ so that the covariances of the $(QX)_{ij}$'s do not depend on $Q$.

$\newcommand\R{\Bbb R}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution, it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed. $\quad\Box$

$\newcommand\R{\Bbb R}\newcommand\Si{\Sigma}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution (see the details on this below), it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed. $\quad\Box$


Details on why $QX$ equals $X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$: Write $X=[X_{ij}]_{i\in[n],j\in[d]}$ and $Q=[Q_{ij}]_{i\in[n],j\in[n]}$, where $[n]:=\{1,\dots,n\}$. Then the entries $(QX)_{ij}=\sum_{k\in[n]}Q_{ik}X_{k,j}$ of the matrix $QX$ are zero-mean jointly normal random variables, and for all $i,i'$ in $[n]$ and $j,j'$ in $[d]$ we have $$E(QX)_{ij}(QX)_{i'j'} =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}EX_{kj}X_{k'j'} \\ =\sum_{k,k'\in[n]}Q_{ik}Q_{i'k'}1(k=k')\Si_{jj'} =\sum_{k\in[n]}Q_{ik}Q_{i'k}\Si_{jj'} =1(i=i')\Si_{jj'},$$ so that the covariances of the $(QX)_{ij}$'s do not depend on $Q$.

Source Link
Iosif Pinelis
  • 127.9k
  • 8
  • 107
  • 229

$\newcommand\R{\Bbb R}$The answer to your both questions is yes, regardless of what the covariance matrix $\Sigma$ is.

Indeed, let $P_X:=X(X^\top X)^{-1}X^\top$, the orthoprojector onto $V_X:=X\R^d$, the column space of the random matrix $X$, so that $P_X\R^n=V_X$. Then for any orthogonal matrix $Q\in\R^{n\times n}$ we have $P_{QX}=QX(X^\top X)^{-1}X^\top Q^\top$ and hence $$V_{QX}=QX(X^\top X)^{-1}X^\top Q^\top\R^n =QX(X^\top X)^{-1}X^\top\R^n=QV_X.$$ Therefore and because $QX$ equals $X$ in distribution, it follows that $V_X$ equals $QV_X$ in distribution, for any orthogonal matrix $Q\in\R^{n\times n}$. So, $V_X$ is uniformly distributed. $\quad\Box$