Question

The people aspiring for the Netflix prize like incremental SVDs. See

1. https://issues.apache.org/jira/browse/MAHOUT-371
2. B.M. Sarwar, G.Karypis, J.A. Konstan, and J. Reidl. Incremental singular value deocmposition algorithms for highly scalable recommender systems. In Proceedings of the Fifth International Conference on Computer and Information Technology (ICCIT), 2002. (ONLINE)
3. M. Brand. Fast online svd revisions for lightweight recommender systems. In Proceedings of the 3rd SIAM International Conference on Data Mining, 2003. and his tech report

Have you tried searching the ACM digital library for parallel SVD or singular value decomposition?

EDIT1 based on new input : See the following two papers by Hall, Marshall and Martin

1. Merging and Splitting Eigenspace Models (Section 3)
2. On Adding and Subtracting Eigenspaces with EVD and SVD