Skip to main content
deleted 43 characters in body
Source Link
Norouzi
  • 362
  • 1
  • 10

After a bit of thinking here is an answer. Because $r_i$ is supposed to be unit norm, one can replace $r_i$ with $\frac{r_i}{\lVert r_i \rVert}$ in the objective and obtain: $$ \mathrm{minimize}~~~~\sum_{i=1}^p \log \Big( \frac{r_i^\mathsf{T}\Sigma r_i}{r_i^\mathsf{T}r_i} \Big)~. $$ Now, taking the derivative of this objective with respect to $r_i$ (forgetting about orthogonality constraints for now), we get: $$ \frac{r_i}{r_i^\mathsf{T}r_i} = \frac{\Sigma r_i}{r_i^\mathsf{T}\Sigma r_i}~. $$ Obviously only eigenvectors of $\Sigma$ satisfy such constraints. Fortunately, they are orthogonal too, so the orthogonality constraints are automatically satisfied. Taking the $p$ smallest eigenvectors of $\Sigma$ when they are positive provides an answer.

After a bit of thinking here is an answer. Because $r_i$ is supposed to be unit norm, one can replace $r_i$ with $\frac{r_i}{\lVert r_i \rVert}$ in the objective and obtain: $$ \mathrm{minimize}~~~~\sum_{i=1}^p \log \Big( \frac{r_i^\mathsf{T}\Sigma r_i}{r_i^\mathsf{T}r_i} \Big)~. $$ Now, taking the derivative of this objective with respect to $r_i$ (forgetting about orthogonality constraints for now), we get: $$ \frac{r_i}{r_i^\mathsf{T}r_i} = \frac{\Sigma r_i}{r_i^\mathsf{T}\Sigma r_i}~. $$ Obviously only eigenvectors of $\Sigma$ satisfy such constraints. Fortunately, they are orthogonal too, so the orthogonality constraints are automatically satisfied. Taking the $p$ smallest eigenvectors of $\Sigma$ when they are positive provides an answer.

Because $r_i$ is supposed to be unit norm, one can replace $r_i$ with $\frac{r_i}{\lVert r_i \rVert}$ in the objective and obtain: $$ \mathrm{minimize}~~~~\sum_{i=1}^p \log \Big( \frac{r_i^\mathsf{T}\Sigma r_i}{r_i^\mathsf{T}r_i} \Big)~. $$ Now, taking the derivative of this objective with respect to $r_i$ (forgetting about orthogonality constraints for now), we get: $$ \frac{r_i}{r_i^\mathsf{T}r_i} = \frac{\Sigma r_i}{r_i^\mathsf{T}\Sigma r_i}~. $$ Obviously only eigenvectors of $\Sigma$ satisfy such constraints. Fortunately, they are orthogonal too, so the orthogonality constraints are automatically satisfied. Taking the $p$ smallest eigenvectors of $\Sigma$ when they are positive provides an answer.

Source Link
Norouzi
  • 362
  • 1
  • 10

After a bit of thinking here is an answer. Because $r_i$ is supposed to be unit norm, one can replace $r_i$ with $\frac{r_i}{\lVert r_i \rVert}$ in the objective and obtain: $$ \mathrm{minimize}~~~~\sum_{i=1}^p \log \Big( \frac{r_i^\mathsf{T}\Sigma r_i}{r_i^\mathsf{T}r_i} \Big)~. $$ Now, taking the derivative of this objective with respect to $r_i$ (forgetting about orthogonality constraints for now), we get: $$ \frac{r_i}{r_i^\mathsf{T}r_i} = \frac{\Sigma r_i}{r_i^\mathsf{T}\Sigma r_i}~. $$ Obviously only eigenvectors of $\Sigma$ satisfy such constraints. Fortunately, they are orthogonal too, so the orthogonality constraints are automatically satisfied. Taking the $p$ smallest eigenvectors of $\Sigma$ when they are positive provides an answer.