Skip to main content
deleted 2 characters in body
Source Link

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one betweenamong the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_i}(x_i)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one between the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_i}(x_i)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one among the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_i}(x_i)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.

edited body
Source Link

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one between the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_k}(x_k)\|^2$$h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_i}(x_i)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one between the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_k}(x_k)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one between the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_i}(x_i)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.

Source Link

This comment is too long so I directly post it as an answer. It is not hard to propose such a method.

Assumption: Every $f_i(x)$ is Lipschitz differentiable with constant $L$, and define $h(x):=\min_{i=1,...,q}f_i(x)$. Suppose further that $h(x)$ is bounded from below.

First, it is easy to prove:

Optimality condition:Assume for every $j$ such that $f_j(z)=h(z)$ we have $\nabla f_j(z)=0$, then $h(x)$ is differentiable at $z$ and $\nabla h(z)=0$.

Second, start from $x_k$, let $f_{j_k}(x_k)=h(x_k)$, and $\|\nabla f_{j_k}(x_k)\|$ is the largest one between the index set such that $f_j(x_k)=h(x_k)$. Compute $x_{k+1}:=x_k-\frac{1}{L}\nabla f_{j_k}(x_k)$. Then you could prove that $h(x_k)\leq h(x_0)-\frac{1}{2L}\sum_{i=0}^{k-1}\|\nabla f_{j_k}(x_k)\|^2$. Push to the limit, it is easy to prove that any cluster point of $x_k$ is a stationary point of $h(x)$.

By the way, in general, I think your problem is a special form of bilevel optimization.