The SVD (singular value decomposition) is taught in many linear algebra courses. It's taken for granted that it's important. I have helped teach a linear algebra course before, and I feel like I need to provide a better motivation for the SVD. This has motivated me to ask what the applications of the SVD might be.

I have considered the following applications, but I am sceptical of some of the ones I list:

*Low-rank approximation in image compression*. Aren't images usually compressed using an algorithm like JPEG which is based on the discrete cosine transform (DCT) and not on low-rank approximation? Basically, is this something that's used in competitive image compression algorithms?*Low-rank approximation in recommender systems*. The problem to solve is that you have a matrix $M$ where the columns are users and the rows are products. The entries of the matrix tell you what rating each user gave to each product. Notice that for many use-cases, some of the entries of the matrix will be missing. Picture those missing entries as taking on the special value "?" (that is, a question mark). I am almost certain that you can't use a default value like $0$. Supposedly, given a low-rank approximation $M \approx U\Sigma V^T$, the columns of $V$ represent products, and columns which have a high cosine-similarity represent similar products. "Cosine-similarity" is simply the cosine of the angle between between vectors. The problem is that you can't take the SVD of a matrix with missing values. This point is well known among practitioners who have developed approaches to low-rank approximation of matrices with missing entries that don't directly involve using the SVD (and I'm not sure how they even*could*use the SVD).*Finding the nearest orthogonal matrix to a given matrix in machine vision*. The objective is, given a matrix $M$, find the orthogonal matrix $\Omega$ for which $\|M - \Omega\|$ is minimised. For some matrix norms, this problem can be solved optimally (in the sense of finding the optimal $\Omega$) using SVD. This is potentially useful in machine vision for turning a matrix which*ought to be*orthogonal, but which isn't because of various sources of error, into one that*is*orthogonal. This is helpful for reconstructing the position of a camera in space based on photographs taken with that camera.*Principal component analysis*(PCA). The objective is to reduce the dimensionality of a dataset in order to plot the data or use it in machine learning algorithms that might not work well with high-dimensional data. The idea here is that you imagine that your data is drawn from a multivariate normal distribution. What PCA does is infer the underlying multivariate normal distribution and approximate it using a lower-dimensional multivariate normal. Note that if a dimension gets scaled in the sample set, then the output of PCA can be changed. In many cases in which PCA is used (most cases at the moment?), the dataset is not actually drawn from a normal distribution; and I don't know how these deviations from normality affect the suitability of PCA.*Linear regression*. My understanding is that SVD can be used to find*Moore-Penrose pseudoinverses*, and Moore-Penrose pseudoinverses can in turn be used to fit linear regression models. It's worth pointing out that linear regression is usually done alongside*regularisation*in order to prevent overfitting. One form of regularised linear regression is called*ridge regression*. There is indeed a way to fit a Ridge regression model using SVD. Some properties of Ridge regression can be studied this way.

I'd love to see applications in pure mathematics or physics as well. The above summarises my investigations into SVD thus far.

6more comments