Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

I am confused between PCA and SVD.

The wikipedia page for PCA has this line. "PCA can be done by eigenvalue decomposition of a data covariance matrix or singular value decomposition of a data matrix, usually after mean centering the data for each attribute."

Does this mean that PCA = SVD of data matrix?

Is there an article/tutorial that explains the difference?

share|improve this question
2  
This question really belongs to math.stackexchange.com However, here is a link to lecture notes that very clearly mention the difference between PCA and SVD (mean-centering; one uses $XX^T$, other other uses $X$) cs.utexas.edu/users/inderjit/courses/dm2001/lecture7.ps –  Suvrit May 11 '11 at 10:31
    
@Suvrit: Thanks. –  user15019 May 11 '11 at 11:34
    
I answered this question here: math.stackexchange.com/questions/3869 –  J. M. May 11 '11 at 12:03

1 Answer 1

PCA is map the data to lower dimensional. In order for PCA to do that it should calculate and rank the importance of features/dimensions. There are 2 ways to do so.

  1. using eigenvalue and eigenvector in covariance matrix to calculate and rank the importance of features
  2. using SVD on covariance matrix to calculate and rank the importance of the features SVD (covariance matrix) = [U S V']

after ranking the features/ dimensions then it will choose the most important ones (k) and map the actual data to k dimension.

in case PCA used SVD to rank the importance of features, then U matrix will have all features ranked, we choose the first k columns which represent the most important one.

to determinate k we can use S matrix.

This is a link that explain to you why PCA can use SVD instead of eigvector/eignvalue

http://math.stackexchange.com/questions/3869/what-is-the-intuitive-relationship-between-svd-and-pca

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.