Why eigenvectors optimize this orthogonally constrained nonlinear minimization problem? Given a $p \times p$ positive definite matrix $\Sigma$, why eigenvectors of $\Sigma$, stacked as columns of a matrix $R \equiv [r_1 \, r_2 \, \ldots \, r_p]$, optimize the following orthogonally constrained minimization problems?
$$
\mathrm{minimize}~~~~\log \det\big( I \odot (R^\mathsf{T}\Sigma R) \big)~~~~~\text{s.t.}~~~~~R^\mathsf{T}R = I~~,
$$
where $\odot$ is Hadamard product, or equivalently,
$$
\mathrm{minimize}~~~~\sum_{i=1}^p \log(r_i^\mathsf{T}\Sigma r_i)~~~~~\text{s.t.}~~~~~r_i^\mathsf{T}r_j = \delta_{ij}~~.
$$
 A: Your minimization problem is equivalent to
\begin{equation*}
\min_{R^TR=I}\quad\prod_{i=1}^p r_i^T\Sigma r_i,
\end{equation*}
and it can be shown (using Hadamard's determinant inequality and some more argumentation) that this minimum overall $p$ orthonormal tuples is achieved by choosing the $r_i$ corresponding to the smallest $p$ eigenvectors.
EDIT Here the details that complete my answer (because it seems that the OP already knew the answer and still posted this question...)
We know that for a semidefinite matrix $\Sigma$ with eigenvalues $\lambda_1,\ldots,\lambda_n$, and any orthonormal $R$,
\begin{equation*}
  \prod_{i=n-p+1}^n \lambda_i \le \det(R^T\Sigma R) \le \prod_{i=1}^p (R^T\Sigma R)_{ii} = \prod_{i=1}^p r_i^T\Sigma r_i.
\end{equation*}
We are trying to minimize the upper, bound and by choosing $R$ to be the matrix of the smallest $p$ eigenvectors, we can actually turn the first inequality into an equality. This proves the claim. The first inequality is a classic result on eigenvalues, while the second inequality is Hadamard's determinant theorem.
A: Because $r_i$ is supposed to be unit norm, one can replace $r_i$ with $\frac{r_i}{\lVert r_i \rVert}$ in the objective and obtain:
$$
\mathrm{minimize}~~~~\sum_{i=1}^p \log \Big( \frac{r_i^\mathsf{T}\Sigma r_i}{r_i^\mathsf{T}r_i} \Big)~.
$$
Now, taking the derivative of this objective with respect to $r_i$ (forgetting about orthogonality constraints for now), we get:
$$
\frac{r_i}{r_i^\mathsf{T}r_i} = \frac{\Sigma r_i}{r_i^\mathsf{T}\Sigma r_i}~.
$$
Obviously only eigenvectors of $\Sigma$ satisfy such constraints. Fortunately, they are orthogonal too, so the orthogonality constraints are automatically satisfied. Taking the $p$ smallest eigenvectors of $\Sigma$ when they are positive provides an answer.
