My question is about the well known and well studied singular value decomposition (SVD). What I am working on right now requires performing an SVD repeatedly on a slowly varying matrix. Since I don't need an exact decomposition at every iteration, I was wondering whether there are any statistical estimations of the SVD. I found a paper on estimating the maximal eigenvalue of a matrix with non-negative values using Markov chains. Are there any more general statistical estimation techniques?
The question does not seem to be completely well-defined: Is your matrix dense? Sparse? Do you need the singular vectors or just the singular values? Do you need ALL the singular values or just the top few? What do you mean by "statistical estiamtion techniques", and why are these relevant? If your matrix is "slowly varying", then perturbation theory should be your friend: look up the formulas in Kato's "Perturbation theory" (p. 79, if memory serves), and apply them to $A^t A.$ Now, I have no idea how much faster this is then computing the SVD from scratch -- this depends on your problem structure (see the questions at the beginning of this answeer...)