Say that we have two matrices $X$ and $Y$ of dimensions $(T \times N)$ with $N < T$ and $rank(X)=rank(Y)=N$. Furthermore, define a $(T \times k)$ dimensional matrix $D$ with $k<N$ and $rank(D)=k$. Then, construct the idempotent matrix $M = I - D(D^\prime D)^{-1}D^\prime$, with $I$ being a $T$-dimensional identity matrix.
Let $|| \cdot ||_2$ denote the spectral norm. Is it true that
$||X^\prime M X - Y^\prime M Y||_2 \leq ||X^\prime X - Y^\prime Y||_2$ ?
Simulations definitely indicate that the above inequality holds, no matter how I randomly generate $X,Y,D$. If the above claim does not hold, is it then at least true that
$||X^\prime M X - Y^\prime M Y||_2 \leq ||X^\prime X - Y^\prime Y||_F$,
where $||\cdot||_F$ represents the Frobenius norm?
My own reasoning only brings me to the following. It is straightforward to show that $||X^\prime M X||_2 = ||M XX^\prime M||_2\leq ||M||_2^2||X^\prime X||_2 \leq ||X^\prime X||_2$, since $M$ is an idempotent matrix with the first $T-k$ eigenvalues equal to 1 and the remaining 2 equal to 0. The same obviously holds for $||Y^\prime M Y||$, but that does not seem sufficient for the claim to hold?
For the Frobenius norm proof, I tried to write it as the trace of the inner product and use the inequality that $tr(AB) \leq ||A||_2\sum_i \sigma_i(B)$, but that got me nowhere.
I hope any of you can help. Please let me know if you need more information.
Thanks!
