MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a copy of Linear Algebra Done Right, which I worked through years ago in college. I have been using that book to refresh my knowledge, but it does not have an applied or computational aspect to it at all. What would be a good follow-up or companion to Linear Algebra Done Right to introduce me to applied linear algebra, both the theory and practical aspects such as numerical considerations?

share|cite|improve this question

closed as off topic by Andy Putman, Andrés Caicedo, Franz Lemmermeyer, Steve Huntsman, Qiaochu Yuan Jan 2 '11 at 14:09

Questions on MathOverflow are expected to relate to research level mathematics within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

This question might be more appropriate at another site, like or one of the other sites listed in the FAQ. – S. Carnahan Jan 2 '11 at 1:47
On the flip side, having more mathematicians face the problem of computing a few concrete numbers is not necessarily a bad thing. ;) – Jiahao Chen Jan 2 '11 at 2:09
up vote 6 down vote accepted

"Numerical Linear Algebra" by Trefethen and Bau is IMO the single best book to start learning from. It is lucidly written, concise and relatively inexpensive. Perhaps its main drawback is an unconventional presentation starting from singular value decomposition (SVD) and presenting the other standard transformations as derivatives of SVD. It worked for me though.

There are many other excellent books out there, but any good book should cover the basics like Gaussian elimination, Cholesky factorization, LU and QR decompositions, Householder reflections and Givens rotations as an absolutely bare minimum. Also essential are applications to solving linear systems, least squares problems and eigenvalue computations. To understand more contemporary algorithms, coverage of Krylov subspace algorithms such as CG and GMRES, as well as sparse matrix algorithms, are considered increasingly important additions to the standard canon above.

P.S. The key to numerical work is to figure out methods to solve problems for special matrices (e.g. diagonal matrices or upper triangular matrices), then figure out ways to transform entire classes of matrices into such special forms. This took me a while to appreciate consciously but apparently this is an observation that is too trivial for many textbook authors to write out explicitly. Trefethen and Bau is nice because it actually tries to motivate the development pedagogically rather than presenting a laundry list of Things You Should Know.

share|cite|improve this answer
I agree this is a good book, but I am not sure your comment that it presents the other standard transformations as derivatives of SVD is true. As far as I remember it was possible skip the SVD chapter on first reading if one so wishes. – timur Jan 2 '11 at 1:54
Just to clarify, I meant 'derivative' in the loose sense. The book definitely presents the other decompositions like QR, LU etc. as special cases of SVD. IIRC the presentation was organized as SVD --> QR --> LU --> Cholesky. I think if you skipped SVD, you had at least to start from the QR part. – Jiahao Chen Jan 2 '11 at 2:03
At any rate, the normal presentation is more like Cholesky --> LU --> QR --> SVD. – Jiahao Chen Jan 2 '11 at 2:05
I think the organization QR --> LU --> Cholesky in their book was very good in that combined with their approach it makes the whole thing somehow unified and more geometric. – timur Jan 2 '11 at 19:42
Ok I checked the book. They start with SVD and use it as theoretical basis for the other decompositions. But an algorithm to construct SVD is presented last. I think my confusion was because of this. – timur Jan 2 '11 at 19:44

Golub and Van Loan's book Matrix Computations is a classic.

share|cite|improve this answer

Gil Strang's "Linear Algebra and its Applications" is another classic.

share|cite|improve this answer

How about Lax's Linear Algebra?

share|cite|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.