# Difference between 'generalized gradient' and 'subgradient' ?

Hi, I am wondering what the difference between 'generalized gradient' and 'subgradient' of a (potentially non-differentiable) convex function 'f' is.

The generalized gradient I am interested in is meant in the sense of the paper http://www.is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/ICML2010-Kim_6519[0].pdf (see page 3, footnote 2).

many thx for any help.

-
for convex functions, the set $\partial_G\phi$ there boils down to the standard subdifferential $\partial \phi$..... –  Suvrit Jan 13 '13 at 17:41

Generalized gradients generalize subdifferentials. Subdifferentials are defined globally, and rely on the simple geometry of convex functions. If in a neighborhood a linear subspace only touches the graph of a convex function without crossing it, then you know for a fact it won't cross the graph later. So the intuition between the subdifferential is that you take each linear subspace of that is strictly below the convex function, and slide it up until it touches it. The subdifferential tells you the relation between the slopes of linear subspaces, and the points on the graph where the linear subspace will touch without crossing.

For a general non-convex function, a linear subspace that touches the graph at one point can cross it again at another point. So you need a definition that works locally, only in a small neighborhood of the point. The definition of generalized gradient gives you that. It's reasonably easy to find information online about it. For example, the page on Clarke generalized derivative at Encyclopedia of Mathematics gives a quick introduction. Clarke's book, Optimization and nonsmooth analysis, is nice. A more recent book reference is Rockafeller and Wets, Variational analysis, which thoroughly covers this and related topics.

I quickly looked over the paper you link to, and I don't see why they need generalized gradients instead of just subdifferentials, though I didn't look too closely.

-
I would also recommend Winfried Schirotzek's very nice book Nonsmooth Analysis, which comprehensively covers many generalized differentials and their relations (in infinite dimensions). –  Christian Clason Jan 14 '13 at 10:22
The paper limited its discussion to convex functions, and does not indeed need generalized gradients. However, the same framework generalizes to nonconvex problems, so we left the formulation in terms of generalized gradients (for future ease) :-) –  Suvrit Jan 14 '13 at 12:26
That makes sense, but you might want to add a note to that effect. –  arsmath Jan 14 '13 at 13:14