Skip to main content
Expanded
Source Link
Brian Rushton
  • 3.4k
  • 8
  • 38
  • 63

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Geometrically, though, the formula makes a lot of sense. If the matrix is diagonalized, we are just expanding or contracting along each axis. Imagine a 2-d version of this problem, with the x-axis expanding and the y-axis contracting (or x expanding faster than y, etc.) If we take $\alpha$ to be along either axis, its image under the transformation points in the same direction. Putting the vector exactly between the two (at a 45 degree angle) maximizes the amount that it bends away from y and towards x.

In higher dimensions, we can do the same trick along each pair of axes, but it is maximized when the two axes are as different as possible.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a fixed positive-definite symmetric matrix.

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a fixed positive-definite symmetric matrix.

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Geometrically, though, the formula makes a lot of sense. If the matrix is diagonalized, we are just expanding or contracting along each axis. Imagine a 2-d version of this problem, with the x-axis expanding and the y-axis contracting (or x expanding faster than y, etc.) If we take $\alpha$ to be along either axis, its image under the transformation points in the same direction. Putting the vector exactly between the two (at a 45 degree angle) maximizes the amount that it bends away from y and towards x.

In higher dimensions, we can do the same trick along each pair of axes, but it is maximized when the two axes are as different as possible.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a fixed positive-definite symmetric matrix.

Changed problem explanation.
Source Link
Brian Rushton
  • 3.4k
  • 8
  • 38
  • 63

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a linear transformationfixed positive-definite symmetric matrix.

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a linear transformation.

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a fixed positive-definite symmetric matrix.

Post Undeleted by Brian Rushton
Post Deleted by Brian Rushton
Source Link
Brian Rushton
  • 3.4k
  • 8
  • 38
  • 63

This is only a partial, heuristic answer, but if you let $A$ be the square root of $\sigma$, then $\cos(\omega)$ is equal to $\frac{\alpha' \alpha}{\sqrt{\alpha' A^{-1}\alpha}\sqrt{\alpha' A\alpha}}$. If we let $\alpha$ be a unit vector, the top will always be one, and the bottom will be how much $A^{-1}$ expands $\alpha$ (which is at most the inverse of the smallest eigenvalue) times how much $A$ expands $\alpha$ (which is at most the largest eigenvalue). Just feeling it out, you probably want to add the minimum and maximum eigenvectors together since it will help make each factor in the denominator small.

Note: to other readers, the OP is trying to minimize the angle between a vector and its image under a linear transformation.