2
$\begingroup$

can someone point me to the direction how to calculate the derivatives of a sum of singular values of a matrix? I am trying to minimize $$\min_A \parallel A \parallel_*+ \cdots $$ where $\parallel A \parallel_*=\sum_i \sigma_i$ is the sum of the singular values of $A$.

This is motivated by this paper.

Thank you very much!

Update: Here is the full problem: $$\min_A \parallel A \parallel_* + \lambda \parallel \nabla_A \parallel D-A-E\parallel_F^2 \parallel_F^2$$ with targeted decomposition $D=A+E$

The authors of the mentioned paper use the so called "singular value thresholding" to solve it:

$$u,s,v=SVD(A-\frac{1}{2}(A+E-D))$$ after setting $S=$diag($(s_i-\tau)_+$) $$A_{k+1}=u\cdot S \cdot v^T$$

However, I dont see how to get there. :(

$\endgroup$
4
  • $\begingroup$ Are you sure you need the derivative? The minimum is obviously $-\infty$, unless you have constraints. It may be easier to help you if you state the problem in full. $\endgroup$ Feb 17, 2014 at 18:53
  • $\begingroup$ @Alex singular values are constrained to be nonnegative, so the minimum is $0$. $\endgroup$ Feb 17, 2014 at 19:19
  • 1
    $\begingroup$ Look at page 5 of arxiv.org/pdf/0810.3286 -- this paper uses the subdifferential theorems of A. Lewis to derive the thresholding operator. A simpler derivation of all of this can also be obtained: to do that see the Moreau decomposition, as discussed e.g., in: arxiv.org/pdf/1204.1437 $\endgroup$
    – Suvrit
    Feb 17, 2014 at 23:43
  • $\begingroup$ Thanks to all of you! I found Suvrit's answers best for my need. $\endgroup$
    – mojovski
    Feb 18, 2014 at 13:43

1 Answer 1

2
$\begingroup$

This function is not differentiable (consider $A=0$). If you are interested in learning about its subdifferential (and more on subdifferential of spectral functions), please refer to the excellent papers:

  1. A. Lewis. Nonsmooth analysis of singular values: Part I
  2. A. Lewis. Nonsmooth analysis of singular values: Part II
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.