Skip to main content
minor edit.
Source Link
Suvrit
  • 28.6k
  • 7
  • 82
  • 150

To lay the question to rest, let me do two things: (i) restate it; (ii) answer it.

By $\|x\|$, we mean the Euclidean 2-norm throughout.

Show that the induced 2-norm $$\max_{\|x\|\not= 0} \frac{\|Ax\|}{\|x\|}$$ is given by $\sqrt{\lambda_{\max}(A^TA)}$

The proof is textbook material. For the lazy, here is an informal sketch.

Notice that since without loss of generality, we may rescale vector $x$, hence we may equivalently consider maximizing $\|Ax\|$ such that $\|x\|=1$.

Consider, $\|Ax\|^2 = x^TA^TAx$. The matrix $A^TA$ is SPD, so it has the eigendecomposition $V\Lambda V^T$, where $\Lambda$ is a nonnegative diagonal matrix. Thus, we have $x^TA^TAx = x^TV\Lambda V^Tx = y^T\Lambda y = \sum_i \lambda_i y_i^2$. This, implies that $\|Ax\|^2 \le \lambda_{\max}y^Ty = \lambda_{\max}x^TV^TVx=\lambda_{\max}$ because $V^TV=I$ and $x^Tx=1$.

To conclude the proof we now need to show that in fact $\|Ax\|^2 = \lambda_{\max}$. But this is trivial, because picking $x=v_{\max}$ (eigenvector corr to max eigenvalue), we attain this equality.

PS: Other proofs based on Lagrange multipliers etc. can also be given, but ultimately one needs to invoke something like $Ax=\lambda x$$A^TAx=\lambda x$ at some point.

To lay the question to rest, let me do two things: (i) restate it; (ii) answer it.

By $\|x\|$, we mean the Euclidean 2-norm throughout.

Show that the induced 2-norm $$\max_{\|x\|\not= 0} \frac{\|Ax\|}{\|x\|}$$ is given by $\sqrt{\lambda_{\max}(A^TA)}$

The proof is textbook material. For the lazy, here is an informal sketch.

Notice that since without loss of generality, we may rescale vector $x$, hence we may equivalently consider maximizing $\|Ax\|$ such that $\|x\|=1$.

Consider, $\|Ax\|^2 = x^TA^TAx$. The matrix $A^TA$ is SPD, so it has the eigendecomposition $V\Lambda V^T$, where $\Lambda$ is a nonnegative diagonal matrix. Thus, we have $x^TA^TAx = x^TV\Lambda V^Tx = y^T\Lambda y = \sum_i \lambda_i y_i^2$. This, implies that $\|Ax\|^2 \le \lambda_{\max}y^Ty = \lambda_{\max}x^TV^TVx=\lambda_{\max}$ because $V^TV=I$ and $x^Tx=1$.

To conclude the proof we now need to show that in fact $\|Ax\|^2 = \lambda_{\max}$. But this is trivial, because picking $x=v_{\max}$ (eigenvector corr to max eigenvalue), we attain this equality.

PS: Other proofs based on Lagrange multipliers etc. can also be given, but ultimately one needs to invoke $Ax=\lambda x$ at some point.

To lay the question to rest, let me do two things: (i) restate it; (ii) answer it.

By $\|x\|$, we mean the Euclidean 2-norm throughout.

Show that the induced 2-norm $$\max_{\|x\|\not= 0} \frac{\|Ax\|}{\|x\|}$$ is given by $\sqrt{\lambda_{\max}(A^TA)}$

The proof is textbook material. For the lazy, here is an informal sketch.

Notice that since without loss of generality, we may rescale vector $x$, hence we may equivalently consider maximizing $\|Ax\|$ such that $\|x\|=1$.

Consider, $\|Ax\|^2 = x^TA^TAx$. The matrix $A^TA$ is SPD, so it has the eigendecomposition $V\Lambda V^T$, where $\Lambda$ is a nonnegative diagonal matrix. Thus, we have $x^TA^TAx = x^TV\Lambda V^Tx = y^T\Lambda y = \sum_i \lambda_i y_i^2$. This, implies that $\|Ax\|^2 \le \lambda_{\max}y^Ty = \lambda_{\max}x^TV^TVx=\lambda_{\max}$ because $V^TV=I$ and $x^Tx=1$.

To conclude the proof we now need to show that in fact $\|Ax\|^2 = \lambda_{\max}$. But this is trivial, because picking $x=v_{\max}$ (eigenvector corr to max eigenvalue), we attain this equality.

PS: Other proofs based on Lagrange multipliers etc. can also be given, but ultimately one needs to invoke something like $A^TAx=\lambda x$ at some point.

Source Link
Suvrit
  • 28.6k
  • 7
  • 82
  • 150

To lay the question to rest, let me do two things: (i) restate it; (ii) answer it.

By $\|x\|$, we mean the Euclidean 2-norm throughout.

Show that the induced 2-norm $$\max_{\|x\|\not= 0} \frac{\|Ax\|}{\|x\|}$$ is given by $\sqrt{\lambda_{\max}(A^TA)}$

The proof is textbook material. For the lazy, here is an informal sketch.

Notice that since without loss of generality, we may rescale vector $x$, hence we may equivalently consider maximizing $\|Ax\|$ such that $\|x\|=1$.

Consider, $\|Ax\|^2 = x^TA^TAx$. The matrix $A^TA$ is SPD, so it has the eigendecomposition $V\Lambda V^T$, where $\Lambda$ is a nonnegative diagonal matrix. Thus, we have $x^TA^TAx = x^TV\Lambda V^Tx = y^T\Lambda y = \sum_i \lambda_i y_i^2$. This, implies that $\|Ax\|^2 \le \lambda_{\max}y^Ty = \lambda_{\max}x^TV^TVx=\lambda_{\max}$ because $V^TV=I$ and $x^Tx=1$.

To conclude the proof we now need to show that in fact $\|Ax\|^2 = \lambda_{\max}$. But this is trivial, because picking $x=v_{\max}$ (eigenvector corr to max eigenvalue), we attain this equality.

PS: Other proofs based on Lagrange multipliers etc. can also be given, but ultimately one needs to invoke $Ax=\lambda x$ at some point.