What is the right citation for the power iteration method to find eigenvalues, if I want to cite the method in a paper? I've seen some Google PageRank references in this context. But Brin and Page didn't invent the power iteration method, did they?
Householder called this Simple Iteration, and attributed the first treatment of it to Müntz (1913). Bodewig attributes the power method to von Mises, and acknowledges Müntz for computing approximate eigenvalues from quotients of minors of the explicitly computed matrix $A^k$ , for increasing values of $k$.
Ortiz and Pinkus also mention in "Herman Müntz: A Mathematician’s Odyssey" that
In 1913 he published two notes in Comptes Rendus in connection with the use of iterative techniques for the solutions of algebraic equations. It is very possible that Müntz was the first to develop an iterative procedure for the determination of the smallest eigenvalue of a positive definite matrix. It certainly predates the more generally quoted result of R. von Mises of 1929.
Müntz's real innovation in the first one of his 1913 papers is the method of simultaneous or orthogonal iteration: "Qu'on parte des n directions consecutives ...; en orthogonalisant et en normant, d'apres les notations connues de M. Schmidt, les determinants, on obtiendra convergence ... vers les axes principaux, en general toutes les n."
The relevant papers by Müntz:
- Solution directe de l’équation séculaire et de quelques problèmes analogues transcendants, C. R. Acad. Sci. Paris, 156 (1913), 43-46.
- Sur la solution des équations séculaires et des équations intégrales, C. R. Acad. Sci. Paris, 156 (1913), 860-862.
In his Ph.D. thesis Erhard Schmidt proved the existence of an eigenfunction of a symmetric integral operator by using iterated kernels, the equivalent of matrix powers in a function space. In order to avoid convergence problems in the case of eigenvalues of equal modulus, he did not apply the integral operator directly to a function.
In his 1909 textbook "Determinantentheorie" Gerhard Kowalewski, a distant relative of Kovalevskaya's husband, gave a finite-dimensional version of Schmidt's existence proof for a symmetric matrix T, and then proposed "a simple infinite process to get to an invariant of T", the power method.
I vaguely remember, that this is due to Hotelling (1933). I don't have the monography of S. Mulaik "The foundations of factor analysis" (1972) at hand, but google-books let's you have a peek for "Hotelling". At page 114/115 this is discussed and Mulaik writes: "However, in 1933 Hotelling published a paper in the Journal of Educational Psychology which described a method of finding the characteristic equation directly (...)" Page 115: "(...) permitting us thereby to pick from among the eigenvectors the one associated with the largest eigenvalue. As a matter of fact, both kinds of methods are available. The first, which converges to the eigenvector associated with the largest eigenvalue, is due to Hotelling.(...)"
The method named after Jacobi to find the eigenvalues/eigenvectors is the special case of rotating the columns of a matrix to approximate a certain maximization criterion iteratively. This can be done if a symmetric matrix was decomposed for instance in its two triangular cholesky-factors, and the lower triangular factor is ("Jacobi"-) rotated to "principal components position". Here all pairs of columns are rotated to maximize some criterion and this is repeated until some convergence criterion is satisfied (I can provide that criterion if needed because I've implemented it in a software).
The book of S. Mulaik is a bit aged and of the year 1972, and although there is a lot of modern development in factor-analysis I rate it as still the best monography/standard textbook for the basic understanding of factor analysis and related basic methods of linear algebra (as well for the history...)