Timeline for Why eigenvectors optimize this orthogonally constrained nonlinear minimization problem?
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 14, 2013 at 21:46 | comment | added | Norouzi | @Hansen: By rectangular R, I mean the general case is which $\Sigma$ is $n \times n$ and R is $n \times p$ ($n > p$). | |
| Nov 14, 2013 at 20:16 | comment | added | Hans | @Norouzi: You are right for the most part. I did not think thoroughly through earlier regarding the convexity of the objective function. Restricted to the unit norm, the set of $R$ is the orthogonal group $O(n)$. It is compact and with no boundary. That is why the everywhere differentiable objective function can not be convex everywhere. By the way, by "rectangular" do you mean orthogonal? Also, it would be nice if you could complete your proof in the answer section with what we have discussed. I will up-vote your answer if you make it whole. | |
| Nov 14, 2013 at 15:59 | comment | added | Norouzi | @Hansen: Good point! I think if we agree that the original objective (without log) is bounded below (it is bounded below by zero because of positive definiteness), and is differentiable everywhere, and is defined on a closed set, then the local and global minimas should be achieved at extrema points. The description above tries to find the extrema points by setting the gradient to zero. The objective is actually not convex at least when the matrix R is rectangular; any subset of eigenvectors is an extrema point. | |
| Nov 13, 2013 at 22:06 | comment | added | Hans | @Norouzi: What you have shown is necessity. Should you not prove the convexity of the objective function in your answer to show the concluding condition is sufficient, too? | |
| Nov 13, 2013 at 21:17 | comment | added | Norouzi | As I mentioned above, I didn't know the answer. After I posted the question, I solved it myself too. I like your answer better than mine though. Thanks! | |
| Nov 13, 2013 at 20:55 | comment | added | Suvrit | looks like a nice proof, but if you already knew the answer how come you asked the question? | |
| Nov 13, 2013 at 20:36 | history | edited | Norouzi | CC BY-SA 3.0 |
deleted 43 characters in body
|
| Nov 13, 2013 at 18:56 | history | answered | Norouzi | CC BY-SA 3.0 |