## How do I approach Optimal Control?

Other than learning basic calculus, I don't really have an advanced background. I was curious to learn about Optimal Control (the theory that involves, bang-bang, Potryagin's Maximum Principle etc.) but any article that I start off with, mentions the following: "Consider a control system of the form..." and then goes on to defining partial differential equations. In short, I am lost.

Can someone suggest me a path I should take to learn more about Optimal Control from the very basics?

-
My field is mathematical programming, and to me, optimal control is just optimization with ODEs in the constraint set, that is, it is the optimization of dynamic systems. I would start by studying some optimization theory and getting an intuitive feel for the motivations behind stationarity and optimality conditions -- that will lead naturally into optimal control theory. – Gilead Oct 29 2010 at 3:31
@Gilead: That's great to hear. Could you point me towards some good optimization theory material in that case? I've been Googling but am not sure if I'm starting with the right ones... – Legend Oct 29 2010 at 3:41
I'm hard pressed to recommend any single book, but here are some good starting points: 1) Nonlinear Programming (Dimitri Bersekas); 2) Convex Optimization (Boyd and Vanderberghe); Also one of the classic OC texts is Applied Optimal Control (Bryson and Ho), but it may be out of print. – Gilead Oct 29 2010 at 3:55
A very good place to start is actually Wikipedia: look up articles on Calculus of Variations, Euler-Lagrange equation, trajectory optimization etc. For optimization theory, look up KKT conditions, and start branching off from there. – Gilead Oct 29 2010 at 4:01

My field is mathematical programming, and I tend to look at optimal control as just optimization with ODEs in the constraint set; that is, it is the optimization of dynamic systems. I would start by studying some optimization theory (not LPs but NLPs) and getting an intuitive feel for the motivations behind stationarity and optimality conditions -- that will lead naturally into optimal control theory.

I should mention there is another facet of optimal control, related to control systems. The systems considered are discrete time (as opposed to continuous in PMP) therefore it's difference equations instead of differential equations. Examples of optimal control laws in this latter sense are Linear Quadratic Regulators (LQRs), Linear Quadratic Gaussian (LQGs), Model Predictive Control (MPC). It is this latter type of optimal control that is actually applied in industry. The Pontryagin principle, while useful for analysis, is generally intractable for real-time application to nontrivial plants.

-

I recommend Eduardo Sontag's Mathematical Control Theory without hesitation. It explains the basics of control theory, optimal control inclusive, as mathematicians see it - geared towards advanced undergrads but useful for all. The easier books to read are for and by engineers - nothing against them, I'm one - but if you want a mathematical text that gives the whole story I suggest you look at Sontag's.

-
Kirk's book mentioned in 28734/why-does-not-the-hamiltonian-depend-on-the-derivative-of-the-state is another standard reference. Still, I think you will find more points of contact with your background in Sontag's book than anywhere else. – Pait Apr 14 2011 at 1:42
Forgot to add: Hector Sussmann's paper "300 years of optimal control: from the brachystochrone to the maximum principle" is a very good overview with references and a historical perspective. Highly recommended. Can be viewed at either link below: ieeexplore.ieee.org/xpls/… math.rutgers.edu/~sussmann/papers/… – Pait May 6 2011 at 13:59

You might want to check Lawrence Evans' notes on optimal control: An Introduction to Mathematical Optimal Control Theory [pdf].

-
These note are VERY nice. Thanks for pointing them out. – PeterR Nov 7 2010 at 12:52

A very good little book on the subject is Analytical Methods of Optimization by D.F. Lawden, available from Dover Press. It covers Pontryagin's principal, Hamiltonian and Lagrangian formulations, and should be accessible to a person with your background.

-

It might help to understand what background you already have. Have you taken any courses in ordinary differential equations? partial differential equations? real analysis? What mathematics courses have you taken? What kind of background do you have in engineering approaches to dynamical systems? Have you taken an introductory course in linear systems? Are you familiar with basic concepts like feedback? stability of a system?

-