Let $X, Y \in \mathbb{R}^{n\times p}$ such that $\Vert X- Y \Vert_{HS} \leq \delta$ (Hilbert-Schmidt norm). Also, assume that both $X, Y$ have full column rank. Let the orthogonal prpjection operator onto the column space of $X$ and $Y$ be $P_X$ and $P_Y$. Thus we have $P_X = X (X^\top X)^{-1} X^\top$ and $P_Y =Y (Y^\top Y)^{-1} Y^\top $. I am trying to upper-bound $\Vert P_X - P_Y\Vert_{HS}$ in terms of $\delta$. Is there any well known bound of such form?

I was trying to solve it for $p=1$ case. In that case, $P_X = XX^\top/\Vert X\Vert^2$ and $P_Y$ is defined similarly. We can consider the functions: $$ \Phi_{ij}(X) = \frac{X_i X_j}{\Vert X\Vert^2}. \quad 1\leq i, j\leq n, $$ These functions are not continuous at $X = 0$. But, still, I would expect $\vert\Phi_{ij}(X) - \Phi_{ij}(Y)\vert \leq f(\delta)$ for some function $f$.

I am actually trying find a covering number for a space of projection matrices $\{P_X: det(X^\top X) \neq 0, X \in \mathcal{M}\}$, where the diameter of the set $\mathcal{M} \subseteq \mathbb{R}^{n \times p}$ with respect to the Hilbert-Schmidt norm $\Vert \cdot \Vert_{HS}$ is at most $\delta$. If I can show the above property, then it would help in deriving the upper bound of the covering number of the space of projection matrices.

Thank you in advance for any help.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.