# How do I approach Optimal Control?

Other than learning basic calculus, I don't really have an advanced background. I was curious to learn about Optimal Control (the theory that involves, bang-bang, Potryagin's Maximum Principle etc.) but any article that I start off with, mentions the following: "Consider a control system of the form..." and then goes on to defining partial differential equations. In short, I am lost.

Can someone suggest me a path I should take to learn more about Optimal Control from the very basics?

-
@Gilead: That's great to hear. Could you point me towards some good optimization theory material in that case? I've been Googling but am not sure if I'm starting with the right ones... – Legend Oct 29 '10 at 3:41
I'm hard pressed to recommend any single book, but here are some good starting points: 1) Nonlinear Programming (Dimitri Bersekas); 2) Convex Optimization (Boyd and Vanderberghe); Also one of the classic OC texts is Applied Optimal Control (Bryson and Ho), but it may be out of print. – Gilead Oct 29 '10 at 3:55
A very good place to start is actually Wikipedia: look up articles on Calculus of Variations, Euler-Lagrange equation, trajectory optimization etc. For optimization theory, look up KKT conditions, and start branching off from there. – Gilead Oct 29 '10 at 4:01

My field is mathematical programming, and I tend to look at optimal control as just optimization with ODEs in the constraint set; that is, it is the optimization of dynamic systems. I would start by studying some optimization theory (not LPs but NLPs) and getting an intuitive feel for the motivations behind stationarity and optimality conditions -- that will lead naturally into optimal control theory.

I should mention there is another facet of optimal control, related to control systems. The systems considered are discrete time (as opposed to continuous in PMP) therefore it's difference equations instead of differential equations. Examples of optimal control laws in this latter sense are Linear Quadratic Regulators (LQRs), Linear Quadratic Gaussian (LQGs), Model Predictive Control (MPC). It is this latter type of optimal control that is actually applied in industry. The Pontryagin principle, while useful for analysis, is generally intractable for real-time application to nontrivial plants.

-

You might want to check Lawrence Evans' notes on optimal control: An Introduction to Mathematical Optimal Control Theory [pdf].

-
These note are VERY nice. Thanks for pointing them out. – PeterR Nov 7 '10 at 12:52

I recommend Eduardo Sontag's Mathematical Control Theory without hesitation. It explains the basics of control theory, optimal control inclusive, as mathematicians see it - geared towards advanced undergrads but useful for all. The easier books to read are for and by engineers - nothing against them, I'm one - but if you want a mathematical text that gives the whole story I suggest you look at Sontag's.

-
Kirk's book mentioned in 28734/why-does-not-the-hamiltonian-depend-on-the-derivative-of-the-state is another standard reference. Still, I think you will find more points of contact with your background in Sontag's book than anywhere else. – Pait Apr 14 '11 at 1:42
Forgot to add: Hector Sussmann's paper "300 years of optimal control: from the brachystochrone to the maximum principle" is a very good overview with references and a historical perspective. Highly recommended. Can be viewed at either link below: ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=588098&tag=1 math.rutgers.edu/~sussmann/papers/systems-magazine-brach.ps.gz – Pait May 6 '11 at 13:59

One potential tactic would be start with Estimation Theory rather than Control Theory. I've enjoyed the approach taken in H. Vincent Poor's An Introduction to Signal Detection and Estimation.

-
@mhum: Thank you. I will go through that and get back. – Legend Oct 29 '10 at 0:58

It might help to understand what background you already have. Have you taken any courses in ordinary differential equations? partial differential equations? real analysis? What mathematics courses have you taken? What kind of background do you have in engineering approaches to dynamical systems? Have you taken an introductory course in linear systems? Are you familiar with basic concepts like feedback? stability of a system?

-
@Brian Borchers: I've taken Undergrad level Calculus. I've read about Partial differential equations in the process. I have taken a course on Control Systems but that was more towards Laplace transforms and Z-Transforms. I don't remember hearing about optimal control anywhere. And yes, I've studied feedback as part of Control Systems where most of it was related to transfer functions and root locus determination. Its been almost 8 years since I've studied those so I'm feeling a little lost now when I look at bang-bang structure and maximum principles. – Legend Oct 29 '10 at 0:58
I'm afraid that my answer would start with "take an introductory real analysis course, an introductory course in systems of ODE's, and an introductory course in optimization", so my advice is probably not going to be helpful to you. My background is in mathematics (and computer science) followed by graduate study in optimization, where there's a completely different collection of material that everyone is expected to know. I'd suggest that you look at books and lecture notes that have been written for engineers interested in learning about this stuff. – Brian Borchers Nov 12 '10 at 1:49

A very good little book on the subject is Analytical Methods of Optimization by D.F. Lawden, available from Dover Press. It covers Pontryagin's principal, Hamiltonian and Lagrangian formulations, and should be accessible to a person with your background.

-

Another great book is "Optimal control theory: An introduction to the theory and its applications" by Peter Falb and Michael Athans, also published by Dover. Also, I would recommend looking at the videos of the edX course "Underactuated Robotics", taught by professor Russ Tedrake of MIT.

-

Kamien and Schartwz (1991) is an extremely complete book on this subject. It can occasionally be oblique, but is otherwise quite helpful.

The Economists' Mathematical Manual contains all of Kamien and Schwartz's results (and much more) in a handy summary format.

-