1
$\begingroup$

To minimize a function $f$, the proximal point method is defined as

$$x_{k+1} := \operatorname*{argmin}_x f(x) + \frac{1}{2\eta}\|x - x_k\|^2.$$

What's the intuition for why we want to use L2 regularization and not some higher order for example, since it seems like all we require is convexity?

Improve this question
$\endgroup$
2
  • $\begingroup$ One reason for adding quadratic regularisation is that it (can) make[s] the inner problem 'more' convex (in the sense of strong / uniform convexity), which makes it appreciably easier to solve in many cases. $\endgroup$
    – πr8
    Commented Oct 19 at 19:33
  • 1
    $\begingroup$ I think one should prefer a penalty term that is strongly convex (so as not to be too forgiving) and and also analytically as simple as possible. For either one of these reasons, you cannot beat the isotropic quadratic penalty. $\endgroup$
    – Iosif Pinelis
    Commented Oct 19 at 23:23

0

Reset to default

You must log in to answer this question.