For a nonconvex function $p_\lambda(\cdot)$, with hyper-parameter $\lambda$, it can be decomposed into two parts that $p_\lambda(t) = \lambda |t| + q_\lambda(t)$, where $q_\lambda(t)$ is the concave component. Particularly, $p_\lambda(t)$ and $q_\lambda(t)$ satisfy the following regularity conditions:
- There exists a constant $\nu$ such that the function $p_{\lambda}(t)$ satisfies $p'_{\lambda}(t) = 0, \forall~ t \geq \nu > 0$.
On the nonnegative real line, $q'_{\lambda}(t)$ is monotone and Lipschitz continuous, i.e., for $t' \geq t$, there exists a constant $\zeta_- \geq 0$ such that $ q'_{\lambda}(t') - q'_{\lambda}(t) \geq -\zeta_- (t'-t)$.
Both function $q_{\lambda}(t)$ and its derivative $q'_{\lambda}(t)$ pass through the origin, i.e., $q_{\lambda}(0) = q'_{\lambda}(0) = 0$.
On the nonnegative real line, $|q'_{\lambda}(t)|$ is upper bounded by $\lambda$, i.e., $|q'_{\lambda}(t)| \leq \lambda$.
For a matrix $\mathbf{M} \in \mathbb{R}^{d_1 \times d_2}$, define the nonconvex function $P_\lambda(\cdot)$ on the singular value vector of $\mathbf{M}$ such that $P_\lambda(\mathbf{M}) = \sum_{i = 1}^{d} p_\lambda(\sigma_i(\mathbf{M}))$, where $d = \min\{d_1, d_2\}$ and $\sigma_i(\mathbf{M})$ is the $i$-th singular value of $\mathbf{M}$.
Suppose the SVD of $\mathbf{M}$ is $\mathbf{M} = \mathbf{U\Sigma V^\top}$, we define the subspace associated with the top-$r$ singular values of $\mathbf{M}$ that $$ \mathcal{M} = \{ \boldsymbol{\Theta} \in \mathbb{R}^{d_1 \times d_2} ~\big|~ \text{row}(\boldsymbol{\Theta}) \subset \mathbf{V}^r ~\text{and}~ \text{col}(\boldsymbol{\Theta}) \subset \mathbf{U}^r\}, $$ where $\mathbf{V}^r$ and $\mathbf{U}^r$ are the sub-matrices of singular vectors associated with the top-$r$ singular vectors. We also define $\mathcal{M}^\perp$ as the complementary subspace of $\mathcal{M}$.
We then define projection of an arbitrary matrix $\mathbf{A}$ into the subspace that $$ \Pi_{\mathcal{M}}(\mathbf{A}) = \mathbf{U U^\top A V V^\top}, $$ $$ \Pi_{\mathcal{M}^\perp}(\mathbf{A}) = \mathbf{A} - \mathbf{U U^\top A V V^\top}. $$
For two matrices $\mathbf{A, B}$, we define the subspaces $\mathcal{A, B}$ associated with the top-$r$ singular values of $\mathbf{A, B}$, I am wondering if the following inequality holds or not: $$ \xi P_\lambda(\Pi_{\mathcal{A}}(\mathbf{B})) - P_\lambda(\Pi_{\mathcal{A}^\perp}(\mathbf{B})) \leq \xi P_\lambda(\Pi_{\mathcal{B}}(\mathbf{B})) - P_\lambda(\Pi_{\mathcal{B}^\perp}(\mathbf{B})), $$ where $\xi$ is a constant. If it doesn't hold in general, under what conditions it will hold?
Thank you,
Agnes