Welcome to MathOverflow

Question

2

3

Hi,

I want to begin learning Calculus of Variations. What texts would MathOverflow recommend? Amazon shows up quite a few options:

http://tinyurl.com/36koaq4

I work on Machine Learning, and that where I intend to apply this.

Thanks!

edited Nov 17 2010 at 14:49

Andrey Rekalo

community wiki

4

When I was a student, I found it rather painful to learn about the calculus of variations. Perhaps things have gotten better, but my impression is that the subject is similar to PDE's in that what you need to learn and use depends very much on the specific application you have in mind. So if there has already been work on using optimization in machine learning (and it appears there has), I recommend working your way through a foundational paper and consulting other references as you go along. At some point, you'll realize which book or papers you need to study more carefully. – Deane Yang Nov 17 2010 at 3:14

1

My first ever encounter with the calculus of variations was a chapter in Simmons's book Differential Equations with Applications and Historical Notes, which was one of my favourite books freshman year. It is by no means comprehensive, but it whets your appetite for the more painful stuff that Deane mentions :) – José Figueroa-O'Farrill Nov 17 2010 at 3:22

Recommendations like this are very difficult to make without knowing more about your studies. I recommend that you ask someone in your institution who knows the topic that you are working on. – Andrew Stacey Nov 17 2010 at 10:06

2

In particular, I know of very few places where the ideas or techniques of calculus of variations are used beyond the calculation of the first and second variations of the functional being optimized. And even these calculations are usually best done from scratch, rather than using the general formulas derived in calculus of variations texts. For example, much if not most of differential geometry is devoted to studying optimal objects for various functionals (length, area, average scalar curvature, etc.), but everything is always worked out from scratch. – Deane Yang Nov 17 2010 at 15:00

But the recommendation by PeterR looks promising. – Deane Yang Nov 17 2010 at 15:04

sfilip · Answer 1 · 2010-11-17 17:15:16Z UTC

I would personally recommend Gelfand and Fomin's "Calculus of variations". It has many advantages:

It is cheap (so if you buy it and don't like it, it's not a big deal)
It is written by good mathematicians, that are broad enough to see connections with many different areas.
The English version has useful exercises, and they're reasonable and with an eye on applications.
It has an appendix on Optimal Control, which I guess might be useful for what you want

Overall, I think this is a good book to have anyways, you'll always want to have a look there even if you get a book that is concerned more directly with applications (although as I said, they already keep an eye on what those ideas are useful for outside pure math)

Anton Geraschenko · Answer 2 · 2010-11-17 18:41:08Z UTC

3

Bruce van Brunt's The Calculus of Variations.

edited Nov 17 2010 at 18:41

Anton Geraschenko♦♦

community wiki

Why was this downvoted? It seems like a reasonable suggestion. – Deane Yang Nov 17 2010 at 18:48

4

@Deane: I think it was because of the surrounding text I edited out (see mathoverflow.net/revisions/46376/list) – Anton Geraschenko♦ Nov 17 2010 at 18:57

@Anton I apologize.But with all the trouble I've had and the hard line the moderators have taken against me,I honestly didn't know if I could post this without trouble. So I made a little joke. While I'm here-for more sophisticated readers who have a background in functional analysis and operator theory, there's Jurgen and Xiangiang-Li Jost's CALCULUS OF VARIATIONS. And of course,there's the classic text I first learned it from: L.C. Young's LECTURES ON THE CALCULUS OF VARIATIONS AND OPTIMAL CONTROL THEORY. The last one is somewhat disorganized,but Young writes so beautifully. – Andrew L Nov 17 2010 at 19:29

Anton, thanks for the explanation. – Deane Yang Nov 17 2010 at 20:37

PeterR · Answer 3 · 2010-11-17 15:02:04Z UTC

Chris Bishop's book "Pattern Recognition and Machine Learning" has some stuff on applying variational methods to machine learning. You might have a look at that and follow his references.

Also, to me, "A Primer on the Calculus of Variations and Optimal Control Theory" by Mike Mesterton-Gibbons looks nice. You can get a sample of it (table of contents, the first few pages) at http://www.ams.org/bookstore-getitem/item=STML-50

Tyler Clark · Answer 4 · 2010-11-17 23:56:29Z UTC

Like most people above, I am not really sure what you are doing with this information. However, after you have looked at the continuous case, you might consider looking at the discrete calculus of variations. [1] (listed below) has a very nice chapter (chapter 8) on the discrete calculus of variations.

[1] Kelley, W. & Peterson, A. (2001). Difference Equations: An Introduction with Applications (2nd Ed.). San Diego, CA: Academic Press.

squarehead · Answer 5 · 2010-11-18 06:50:33Z UTC

Thank you everyone for your recommendations and suggestions. As pointed out by some I was, perhaps, not very specific about my needs. I am working with ML and my need to understand the Calculus of Variations came up when I began looking at LDA (www.cs.princeton.edu/~blei/papers/BleiNgJordan2003.pdf).

@PeterR - I own a copy of Bishop, but I needed something more comprehensive in way of being introduced to this subject. I feel the coverage of the topic in the book is highly tailored towards its applications in it. I would like to be in a position apply it on my own models sometime in the near future - so I want to build a good foundation for now. Your other suggestion seems promising from a cursory look - Thanks!

@Deane Yang - "I know of very few places where the ideas or techniques of calculus of variations are used beyond the calculation of the first and second variations of the functional being optimized" --- I pretty much suspect the same from what I have seen so far :)

@sfilip - I have begun looking at Gelfland and Fomin and it does look promising. Thanks!

And again, thanks a lot everyone for your suggestions!

Beginners text on Calculus of Variations

Remember to vote up questions/answers you find interesting or helpful (requires 15 reputation points)

5 Answers

You can accept an answer to one of your own questions by clicking the check mark next to it. This awards 15 reputation points to the person who answered and 2 reputation points to you.