2
$\begingroup$

can someone point me to the direction how to calculate the derivatives of a sum of singular values of a matrix? I am trying to minimize $$\min_A \parallel A \parallel_*+ \cdots $$ where $\parallel A \parallel_*=\sum_i \sigma_i$ is the sum of the singular values of $A$.

This is motivated by this paper.

Thank you very much!

Update: Here is the full problem: $$\min_A \parallel A \parallel_* + \lambda \parallel \nabla_A \parallel D-A-E\parallel_F^2 \parallel_F^2$$ with targeted decomposition $D=A+E$

The authors of the mentioned paper use the so called "singular value thresholding" to solve it:

$$u,s,v=SVD(A-\frac{1}{2}(A+E-D))$$ after setting $S=$diag($(s_i-\tau)_+$) $$A_{k+1}=u\cdot S \cdot v^T$$

However, I dont see how to get there. :(

Improve this question
$\endgroup$
4
  • $\begingroup$ Are you sure you need the derivative? The minimum is obviously $-\infty$, unless you have constraints. It may be easier to help you if you state the problem in full. $\endgroup$
    – Alex Degtyarev
    Commented Feb 17, 2014 at 18:53
  • $\begingroup$ @Alex singular values are constrained to be nonnegative, so the minimum is $0$. $\endgroup$
    – Federico Poloni
    Commented Feb 17, 2014 at 19:19
  • 1
    $\begingroup$ Look at page 5 of arxiv.org/pdf/0810.3286 -- this paper uses the subdifferential theorems of A. Lewis to derive the thresholding operator. A simpler derivation of all of this can also be obtained: to do that see the Moreau decomposition, as discussed e.g., in: arxiv.org/pdf/1204.1437 $\endgroup$
    – Suvrit
    Commented Feb 17, 2014 at 23:43
  • $\begingroup$ Thanks to all of you! I found Suvrit's answers best for my need. $\endgroup$
    – mojovski
    Commented Feb 18, 2014 at 13:43

1 Answer 1

Reset to default
2
$\begingroup$

This function is not differentiable (consider $A=0$). If you are interested in learning about its subdifferential (and more on subdifferential of spectral functions), please refer to the excellent papers:

  1. A. Lewis. Nonsmooth analysis of singular values: Part I
  2. A. Lewis. Nonsmooth analysis of singular values: Part II
Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .