Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

I am confused between PCA and SVD.

The wikipedia page for PCA has this line. "PCA can be done by eigenvalue decomposition of a data covariance matrix or singular value decomposition of a data matrix, usually after mean centering the data for each attribute."

Does this mean that PCA = SVD of data matrix?

Is there an article/tutorial that explains the difference?

share|improve this question

put on hold as off-topic by Lucia, Suvrit, Chris Godsil, Stefan Waldmann, András Bátkai 2 days ago

This question appears to be off-topic. The users who voted to close gave these specific reasons:

  • "MathOverflow is for mathematicians to ask each other questions about their research. See Math.StackExchange to ask general questions in mathematics." – Lucia, Stefan Waldmann, András Bátkai
  • "This question does not appear to be about research level mathematics within the scope defined in the help center." – Suvrit, Chris Godsil
If this question can be reworded to fit the rules in the help center, please edit the question.

This question really belongs to math.stackexchange.com However, here is a link to lecture notes that very clearly mention the difference between PCA and SVD (mean-centering; one uses $XX^T$, other other uses $X$) cs.utexas.edu/users/inderjit/courses/dm2001/lecture7.ps –  Suvrit May 11 '11 at 10:31
@Suvrit: Thanks. –  user15019 May 11 '11 at 11:34
I answered this question here: math.stackexchange.com/questions/3869 –  J. M. May 11 '11 at 12:03

1 Answer 1

PCA is map the data to lower dimensional. In order for PCA to do that it should calculate and rank the importance of features/dimensions. There are 2 ways to do so.

  1. using eigenvalue and eigenvector in covariance matrix to calculate and rank the importance of features
  2. using SVD on covariance matrix to calculate and rank the importance of the features SVD (covariance matrix) = [U S V']

after ranking the features/ dimensions then it will choose the most important ones (k) and map the actual data to k dimension.

in case PCA used SVD to rank the importance of features, then U matrix will have all features ranked, we choose the first k columns which represent the most important one.

to determinate k we can use S matrix.

This is a link that explain to you why PCA can use SVD instead of eigvector/eignvalue


share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.