Suppose that X and Z are matrices with the same number of rows. Let $$ D = \left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1} - \left[\begin{array}{cc} (X' X)^{-1} & 0 \\ 0 & 0 \end{array} \right],$$ where all inverses are assumed to exist and the zeros represent zero matrices of suitable dimensions. How can we prove that $D$ is positive semidefinite?
1 Answer
Let
$$A:=X' X,\quad B:=X'Z,$$
\begin{equation}
\left[\begin{array}{cc} U & V \\ V' & T \end{array} \right]:=
\left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1}.
\end{equation}
Then $AU+BV'=I$, $AV+BT=0$, whence
\begin{equation}
V=-A^{-1}BT,\quad U=A^{-1}+A^{-1}BTB'A^{-1},
\end{equation}
\begin{equation}
D=\left[\begin{array}{cc} A^{-1}BTB'A^{-1} & -A^{-1}BT \\ -(A^{-1}BT)' & T \end{array} \right].
\end{equation}
Also, $T$ is positive definite, as a diagonal block of the positive definite matrix $\left[\begin{array}{cc} X' X & X'Z \\ Z'X & Z'Z \end{array} \right]^{-1}$.
Therefore, for any column matrix $\left[\begin{array}{cc} x \\ z \end{array} \right]$ with sub-columns $x$ and $z$ of appropriate heights, straightforward calculations yield
\begin{equation}
\left[\begin{array}{cc} x \\ z \end{array} \right]'D\left[\begin{array}{cc} x \\ z \end{array} \right]
=(w-z)'T(w-z)\ge0,
\end{equation}
where
$w:=B'A^{-1}x$.
Thus, $D$ is indeed nonnegative definite.