For a nonconvex function $p_\lambda(\cdot)$, with hyper-parameter $\lambda$, it can be decomposed into two parts that $p_\lambda(t) = \lambda |t| + q_\lambda(t)$, where $q_\lambda(t)$ is the concave component. Particularly, $p_\lambda(t)$ and $q_\lambda(t)$ satisfy the following regularity conditions:

  1. There exists a constant $\nu$ such that the function $p_{\lambda}(t)$ satisfies $p'_{\lambda}(t) = 0, \forall~ t \geq \nu > 0$.
  2. On the nonnegative real line, $q'_{\lambda}(t)$ is monotone and Lipschitz continuous, i.e., for $t' \geq t$, there exists a constant $\zeta_- \geq 0$ such that $ q'_{\lambda}(t') - q'_{\lambda}(t) \geq -\zeta_- (t'-t)$.

  3. Both function $q_{\lambda}(t)$ and its derivative $q'_{\lambda}(t)$ pass through the origin, i.e., $q_{\lambda}(0) = q'_{\lambda}(0) = 0$.

  4. On the nonnegative real line, $|q'_{\lambda}(t)|$ is upper bounded by $\lambda$, i.e., $|q'_{\lambda}(t)| \leq \lambda$.

For a matrix $\mathbf{M} \in \mathbb{R}^{d_1 \times d_2}$, define the nonconvex function $P_\lambda(\cdot)$ on the singular value vector of $\mathbf{M}$ such that $P_\lambda(\mathbf{M}) = \sum_{i = 1}^{d} p_\lambda(\sigma_i(\mathbf{M}))$, where $d = \min\{d_1, d_2\}$ and $\sigma_i(\mathbf{M})$ is the $i$-th singular value of $\mathbf{M}$.

Suppose the SVD of $\mathbf{M}$ is $\mathbf{M} = \mathbf{U\Sigma V^\top}$, we define the subspace associated with the top-$r$ singular values of $\mathbf{M}$ that $$ \mathcal{M} = \{ \boldsymbol{\Theta} \in \mathbb{R}^{d_1 \times d_2} ~\big|~ \text{row}(\boldsymbol{\Theta}) \subset \mathbf{V}^r ~\text{and}~ \text{col}(\boldsymbol{\Theta}) \subset \mathbf{U}^r\}, $$ where $\mathbf{V}^r$ and $\mathbf{U}^r$ are the sub-matrices of singular vectors associated with the top-$r$ singular vectors. We also define $\mathcal{M}^\perp$ as the complementary subspace of $\mathcal{M}$.

We then define projection of an arbitrary matrix $\mathbf{A}$ into the subspace that $$ \Pi_{\mathcal{M}}(\mathbf{A}) = \mathbf{U U^\top A V V^\top}, $$ $$ \Pi_{\mathcal{M}^\perp}(\mathbf{A}) = \mathbf{A} - \mathbf{U U^\top A V V^\top}. $$

For two matrices $\mathbf{A, B}$, we define the subspaces $\mathcal{A, B}$ associated with the top-$r$ singular values of $\mathbf{A, B}$, I am wondering if the following inequality holds or not: $$ \xi P_\lambda(\Pi_{\mathcal{A}}(\mathbf{B})) - P_\lambda(\Pi_{\mathcal{A}^\perp}(\mathbf{B})) \leq \xi P_\lambda(\Pi_{\mathcal{B}}(\mathbf{B})) - P_\lambda(\Pi_{\mathcal{B}^\perp}(\mathbf{B})), $$ where $\xi$ is a constant. If it doesn't hold in general, under what conditions it will hold?

Thank you,



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.