## There must be a good introductory numerical analysis course out there!

Background As a numerical analyst, I've frequently taught the 'Introductory Numerical Analysis' class. Such courses are found in many major universities; the audience typically consists of reluctant engineering majors and some majors of mathematics.

The structure of the course is very similar in many of the institutions whose syllabi I've looked at: one begins with finite-precision arithmetic, then fixed-point methods for root-finding (usually 1-D problems),interpolation by polynomials, quadrature, numerical differentiation, some standard ODE methods, and perhaps some finite difference methods for PDE. Any rationale for this particular sequence of topics is obscured in the course.

The truly deep and interesting aspects - approximation theory, error analysis, computational complexity - are either not discussed, or not dwelt on. Instead, the typical introductory course is a collection of algorithms for problems which seem contrived. This is a pity. The stronger mathematics student comes away believing numerical analysis is boring and shallow, and the engineer comes away thinking mathematics has nothing to offer a real problem.

The question: Are there examples (links to course outlines or course webpages preferred) of introductory numerical analysis courses which avoid the above-described tedium, and which have a history of attracting strong mathematics students?

The constraints: The courses should be aimed at students with a background in multivariate calculus, linear algebra, undergraduate dynamical systems and PDE. One example per answer, please.

The motivation: The eventual goal is to compile such a list, and based on these courses suggest a better curriculum at my institution.

-
I am not really qualified to judge, but do any of the notes at damtp.cam.ac.uk/user/na/na.html do some of what you hope for? – Yemon Choi May 19 2011 at 2:44
Thanks- the notes by Iserles are indeed lovely, and are aimed at the students preparing for the Cambridge Tripos. – Nilima Nigam May 19 2011 at 2:50
Is it the case that all the "truly deep and interesting aspects" of numerical analysis are too complicated, or long, to explain in an ordinary course? e.g. how do meteorologists/computational physicists/etc. solve huge systems of equations? Is it really just using the same algorithms that we see in the books, but with expensive supercomputers, or are there fundamentally better techniques which are too difficult to cover? – Zen Harper May 19 2011 at 3:08
Zen, I'd aver that some deep ideas are well within the reach of these students. For example, most of them have seen Fourier series in their PDE courses, and are thus familiar with notions of projection and convergence. Orthogonality is also familiar as a concept. A numerical analysis course would be a neat place to introduce the importance of these notions in the construction of algorithms. There are indeed fundamentally better algorithms out there, some of which we should be introducing earlier. Why wait before describing the FFT? – Nilima Nigam May 19 2011 at 3:45
One problem in teaching such courses is that neither group (engineering students or mathematics majors) is likely to have an adequate background in computer science. This makes it extremely difficult if not impossible to talk about computational complexity in such a course. It also makes it hard to do much practical work on problems of real world size and scope. – Brian Borchers May 19 2011 at 4:37

John Hubbard tends to take sort of the opposite track, in that he likes to bring a more serious numerical analysis perspective into the 1st and 2nd courses on calculus and differential equations, rather than assuming the students come out of a standard service-stream calculus, differential equations, linear algebra sequence of courses. Usually this includes a discussion of various ways of representing numbers on computers, like floating-point numbers, round-off errors, perhaps even topics like interval arithmatic.

For example, once the idea of ODEs are set up he likes to talk about "fences". I don't know if this is standard terminology anywhere or just his, but it's basically like a Lyapanov function but for time-dependent ODEs. So it gives you regions that trap orbits, but the region may move with time. He gets students used to thinking in this way gradually, by cooking up fences in the 1-dimensional time-dependent ODE case first. Then he moves on to things like the Gronwall inequality, applying it for things like the Euler approximations to ODE solutions to observe error growth rates. He also proves Kantorovich's theorem, which he uses for the implicit and inverse function theorems. He has quite a lot of success getting 1st and 2nd year physics and engineering students thinking about these things. But it's known as the "challenging" calculus stream at Cornell, and less ambitious students have other options. I don't know what their numbers are now, but when I was a TA for the course I think he was getting around 80 students per year in the course.

-
 This sounds like a fantastic way to introduce the subject. Do you happen to recall the course number? – Nilima Nigam May 24 2011 at 16:53 Humm, it looks like their course numbers may have changed since I was last there. But from a scan of their webpages it looks like the new course number is Math 2240. See: math.cornell.edu/~hubbard/vectorcalculus.html also he frequently brings in ODEs material from his books with Beverly West. – Ryan Budney May 24 2011 at 17:59

I believe our Numerics course was very interesting. Basically we had the reverse order of the structure in your example.

Numerics (1-Year-Course)

1. Motivational example, heat transfer between two Points. We discretized the Problem and derived a way to solve it (1-dimensional FDM). From this, we then moved on to multiple dimensions and time dependency (FTCS, etc..), introducing error estimates along the way.

2. As obviously each problem boils down to linear equations, we looked at a few of the different iterative algorithms (Gradient, CG, Multigrid...) and of course error estimates and Matrix conditions.

3. We then went on to Interpolation methods, Splines and Co., to replace our linear Ansatz from before.

4. Next, we looked at Quadrature. Even without motivation, it was clear to us that this was usefull

5. At this point, we were able to take short detours and look at different other fields of Numerics briefly, for example, Finite Volume Method. We also took a look at things we had left out, like Newton Method (alot of which were introduced in other lectures).

6. We finalized the course with the Finite Element Method (as this is a core research field at our University), starting with Ritz-Galerkin and ending at a-posteriori error estimates. (Althoug this would need some basic knowledge in Functional Analysis)

I'm the kind of student that will follow a lecture with alot of interest if there is a strong/reasonable motivation behind it. Or at least some sort of "big-picture".

Perhaps to your liking, we had a heavy emphasize on Error estimates. We had alot of real life/hands on examples in between highlighting how important this is. (http://www.ima.umn.edu/~arnold/disasters/sleipner.html)

Further, I have to point out (as briefly mentioned in point 5), that alot of things were already introduced in some other lectures. Mainly our physics lectures required some basic Numerics, so this was not a complete introduction to Numerics.

-
You lament that the notes are in German, and then don't provide a link. Are they on the web? Reading German should be tractable for anyone who knows the subject and just wants another perspective. – Jerry Gagelman May 19 2011 at 8:28
Sadly I only have them in printed form, I'll remove that note from my answer. – Michael Kissner May 19 2011 at 8:32
Michael, this course certainly sounds very interesting! I can read German - would you mind posting a link to the course website? And what references did you use? – Nilima Nigam May 19 2011 at 15:18
Here is a list of references we used: unibw.de/bauv1/lehre/infnumerik/material The structure of the lecture has changed since I've heard it though: Part 1) unibw.de/bauv1/lehre/numerik1me/index_html (More precisely this: unibw.de/bauv1/lehre/numerik1me/ergaenzung) Part 2) unibw.de/bauv1/lehre/infnumerik – Michael Kissner May 19 2011 at 16:22
Danke sehr, Michael ! – Nilima Nigam May 19 2011 at 16:49

When I took a course on numerical analysis a couple of years ago I very much liked the book "An introduction to numerical analysis" by Suli and Mayers, it is very clear and concise. In particular it contains a lot of rigorous error estimates.

-
Thanks- I know and like the book a lot! What would be useful is a link to a one-semester undergraduate course based on it. – Nilima Nigam May 20 2011 at 2:15
The course website is staff.science.uva.nl/~rstevens/numwisk12010.html It's in Dutch but it contains the numbers of the sections covered. The parts of the course I liked most were Gauss quadrature and the Runge phenomenon. – MRB May 20 2011 at 16:11
Both Gaussian quadrature and the Runge phenomenon are great to include, for many reasons. I also like to first present an example and then prove, the excellent performance of the Trapezoidal rule on smooth periodic functions, when integrating over the period. For example, $\int_0^{2\pi} \exp{cos(x)} \,dx$ is approximated well, but $\int_0^{\pi} \exp{cos(x)} \,dx$ is not. – Nilima Nigam May 21 2011 at 1:17

If you haven't, go to the library and take a look at this book: "Numerical Analysis: A Mathematical Introduction", Michelle Schatzman.

It will give you some ideas how to make students fall in love with numerical analysis.

-
 There are lots of excellent books out there, but unfortunately the standard undergraduate NA course in North America is not. Do you know if someone's used the Schatzman book in a course? – Nilima Nigam Jun 15 2011 at 14:53 Sorry, I do not. I just thought it would help to have this at your side if you are going to construct such a course. – Orr Shalit Jun 18 2011 at 11:32

Numerical analysis is a big subject... Stephen Boyd's Convex Optimization (available for download on his web page, or in two pound form from CUP) is a very lucid book, covering both applications and theory.

-
Indeed, Igor, numerical analysis is a big field! Analogously, even though analysis is gigantic, the first analysis course usually manages to introduce important ideas in an elegant framework, which we all tend to follow. Thanks for the suggestion of Boyd's book. Do you know of a course designed for undergrads based on this book? The preface describes it as aimed at the introductory graduate level. – Nilima Nigam May 19 2011 at 3:51
Boyd's webpage: stanford.edu/~boyd has links to several courses, where a lot of the material relevant to the book is covered. I would love to take part in a an effort to design a nice undergraduate level course on numerical optimization. Also note, that math undergrads who are happy with Python might then be able to benefit from tools like CVX, CVXOPT, CVXMOD, etc.--which can also be used to tackle many of the exercises in Boyd and Vandenberghe's book. I also like Lieven's course material, e.g. ee.ucla.edu/~vandenbe/ee103.html – S. Sra May 19 2011 at 8:56
Sorry, I meant to link to: ee.ucla.edu/~vandenbe – S. Sra May 19 2011 at 8:57
Whether the course would be suitable for undergraduates is in the eye of the beholder, but it certainly requires no fancy prerequisites... – Igor Rivin May 19 2011 at 15:21