9
$\begingroup$

Hi, I am wondering what the difference between 'generalized gradient' and 'subgradient' of a (potentially non-differentiable) convex function 'f' is.

The generalized gradient I am interested in is meant in the sense of the paper http://www.is.tuebingen.mpg.de/fileadmin/user_upload/files/publications/ICML2010-Kim_6519[0].pdf (see page 3, footnote 2).

many thx for any help.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ for convex functions, the set $\partial_G\phi$ there boils down to the standard subdifferential $\partial \phi$..... $\endgroup$
    – Suvrit
    Commented Jan 13, 2013 at 17:41

1 Answer 1

Reset to default
10
$\begingroup$

Generalized gradients generalize subdifferentials. Subdifferentials are defined globally, and rely on the simple geometry of convex functions. If in a neighborhood a linear subspace only touches the graph of a convex function without crossing it, then you know for a fact it won't cross the graph later. So the intuition between the subdifferential is that you take each linear subspace of that is strictly below the convex function, and slide it up until it touches it. The subdifferential tells you the relation between the slopes of linear subspaces, and the points on the graph where the linear subspace will touch without crossing.

For a general non-convex function, a linear subspace that touches the graph at one point can cross it again at another point. So you need a definition that works locally, only in a small neighborhood of the point. The definition of generalized gradient gives you that. It's reasonably easy to find information online about it. For example, the page on Clarke generalized derivative at Encyclopedia of Mathematics gives a quick introduction. Clarke's book, Optimization and nonsmooth analysis, is nice. A more recent book reference is Rockafeller and Wets, Variational analysis, which thoroughly covers this and related topics.

I quickly looked over the paper you link to, and I don't see why they need generalized gradients instead of just subdifferentials, though I didn't look too closely.

Improve this answer
$\endgroup$
4
  • $\begingroup$ I would also recommend Winfried Schirotzek's very nice book Nonsmooth Analysis, which comprehensively covers many generalized differentials and their relations (in infinite dimensions). $\endgroup$
    – Christian Clason
    Commented Jan 14, 2013 at 10:22
  • $\begingroup$ The paper limited its discussion to convex functions, and does not indeed need generalized gradients. However, the same framework generalizes to nonconvex problems, so we left the formulation in terms of generalized gradients (for future ease) :-) $\endgroup$
    – Suvrit
    Commented Jan 14, 2013 at 12:26
  • $\begingroup$ That makes sense, but you might want to add a note to that effect. $\endgroup$
    – arsmath
    Commented Jan 14, 2013 at 13:14
  • $\begingroup$ I cannot see how the clarke gradients are local in comparison to the subgradient. There is still some linear form that need to be lower then some directioal derivative. You mind elaborating? $\endgroup$
    – user123124
    Commented Jul 13, 2018 at 6:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .