Let $A$ be a symmetric real-valued $n\times n$ matrix and let ${\left\|A\right\|_{2\rightarrow 2}} := \max_{\left\|u\right\|_{2}\leq 1} \left\|Au\right\|_{2}$ denotes its operator norm (largest singular value). Furthermore, for a strictly positive scalar $\lambda$, define the soft thresholding function by

$$f_{\lambda}(x) := (\left\lvert x \right\rvert-\lambda)_{+} \mathop{\mathrm{sign}}(x) = \max(\left\lvert x \right\rvert-\lambda, 0) \mathop{\mathrm{sign}}(x).$$

Finally, I define the soft thresholded version of $A$ by $A_{\lambda} := \Big[f_{\lambda}(A_{ij})\Big]^n_{i,j=1}$.

Is it true to say that the soft thresholding operation reduces the operator norm, i.e. ${\left\|A_{\lambda}\right\|_{2\rightarrow 2}} \leq {\left\|A\right\|_{2\rightarrow 2}}$. Note that this claim is obviously true for all entry-wise norms such as Frobenius or $\sup$ norms.

I could not find any counter-example for $10^5$ randomly generated Gaussian matrices!

  • $\begingroup$ I don't think so: imagine a large matrix of all 1's with the (1,1) entry replaced by -1.1. The vector corresponding to the largest singular vector will be all positive. Changing -1.1 to -1 increases the norm of the image. $\endgroup$ – Anthony Quas Jan 1 at 19:45
  • $\begingroup$ Thanks for your comment. The soft thresholding operator shrinks all the entries at the same time, right? So, I don't understand how does you example fits into the framework. $\endgroup$ – Student Jan 1 at 19:59
  • $\begingroup$ @AnthonyQuas Mathematica indicates that for $n=50$ the eigenvalues decrease. Here is the relevant code (N[Eigenvalue[ConstantArray[1,{50,50}]-2.1*SparseArray[{{1,1}->1},{50,50}]]], N[Eigenvalue[ConstantArray[.9,{50,50}]-1.9*SparseArray[{{1,1}->1},{50,50}]]]). It is certainly true if the entries are all positive (and greater than $\lambda$) in $A$. $\endgroup$ – Josiah Park Jan 1 at 19:59
  • $\begingroup$ @JosiahPark Yes, the claim is true if all the entries are non-negative. I don't think we need the part regarding greater than $\lambda$. When all the entries are non-negative then all the entries of the leading eigenvector has to be non-negative as well. So, shrinking operation reduces the norm. $\endgroup$ – Student Jan 1 at 20:12
  • 1
    $\begingroup$ It is true for $n=2$, but it is likely you have already checked the trivial cases. $\endgroup$ – Josiah Park Jan 2 at 20:37

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.