Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

I've searched Google, but it seems that only research journal papers appear in search results, where some new, improved, or specialized extragradient method is discussed. I've also searched Wikipedia and Wolfram MathWorld.

I would like to perhaps know the straightforward definition of that, instead of deducing it from a number of articles, where the basic/original method is not defined.

share|improve this question

3 Answers 3

up vote 2 down vote accepted

This is the key reference: G.M. Korpelevich, "The extragradient method for finding saddle points and other problems." Ekonomika i Matematicheskie Metody 12 (1976): 747-756.

I have not found this article online, but you can find a brief description here (page 1 and 2) and a more extensive description (with a convergence proof) here (pages 1-6). Volume II of Facchinei and Pang also has a chapter on the method.

share|improve this answer

I can confirm that there is no agreement of what "extragradient method" really means. I know the interpretation by Christian Clason but I also know the one that is linked in Carlo Beenakkers answer. Let me add my point of view:

For simplicity consider a convex minimization problem $$ \min_{x\in C} F(x) $$ with a convex , lower semi-continuous function $F$ and a convex, closed set $C$.

Optima $u^*$ of this problem are characterized by $$ u^* = P_C(u^* - \sigma\nabla F(u^*)) $$ where $P_C$ is the projection onto $C$ and $\sigma>0$. Doing a fixed point iteration for this equation gives $$ u^{n+1} = P_C(u^n - \sigma\nabla F(u^n)), $$ which is the standard projected gradient method.

Now we introduce a new variable $\bar u$ in the optimality condition and write it as $$ u^* = P_c(u^* - \sigma\nabla F(\bar u)),\qquad u^* = \bar u. $$ This seems artificial but now we can devise a new iteration and decide on different update rules for the artificial variable $\bar u$. The projected gradient method just takes $\bar u^{n+1} = u^{n+1}$. The extragradient method uses another extra gradient step for the new variable and reads as $$ u^{n+1} = P_C(u^n - \sigma\nabla F(\bar u^n)),\qquad \bar u^{n+1} = P_C(u^{n+1} - \sigma \nabla F(\bar u^n)) $$ The benefit of this approach is that, although you make a new gradient step and usually have "better convergence", you have to evaluate the gradient of $F$ only once per iteration (but have to do two projections). If the cost for the evaluation of the gradient dominates the cost for the projection the, you may gain something.

A similar motivation for extragradient methods can be done for methods for variational inequalities, monotone inclusions or saddle-point problems…

share|improve this answer

Briefly, extragradient methods include an extrapolation step for the evaluation of the gradient for the next iteration, e.g., $$ \begin{aligned} \bar x^{k} &= x^k + \tau(x^k - x^{k-1}),\\ x^{k+1} &= x^k + \gamma_k \nabla f(\bar x^k), \end{aligned} $$ where $\gamma_k>0$ is a suitable step size and $\tau\in (0,1)$ is an extrapolation parameter.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.