Positive semidefinite ordering for covariance matrices

Question

Suppose that X and Z are matrices with the same number of rows. Let $$ D = \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1} - \left[\begin{array}{cc} (X' X)^{-1} & 0 \\ 0 & 0 \end{array} \right],$$ where all inverses are assumed to exist and the zeros represent zero matrices of suitable dimensions. How can we prove that $D$ is positive semidefinite?

Answer 1

Let $$A:=X' X,\quad B:=X'Z,$$
\begin{equation} \left[\begin{array}{cc} U & V \\ V' & T \end{array} \right]:= \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1}. \end{equation} Then $AU+BV'=I$, $AV+BT=0$, whence \begin{equation} V=-A^{-1}BT,\quad U=A^{-1}+A^{-1}BTB'A^{-1}, \end{equation} \begin{equation} D=\left[\begin{array}{cc} A^{-1}BTB'A^{-1} & -A^{-1}BT \\ -(A^{-1}BT)' & T \end{array} \right]. \end{equation} Also, $T$ is positive definite, as a diagonal block of the positive definite matrix $\left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1}$. Therefore, for any column matrix $\left[\begin{array}{cc} x \\ z \end{array} \right]$ with sub-columns $x$ and $z$ of appropriate heights, straightforward calculations yield \begin{equation} \left[\begin{array}{cc} x \\ z \end{array} \right]'D\left[\begin{array}{cc} x \\ z \end{array} \right] =(w-z)'T(w-z)\ge0, \end{equation} where $w:=B'A^{-1}x$. Thus, $D$ is indeed nonnegative definite.