Good afternoon, I am studying the book Nonlinear Programming: Theory and Algorithms (by Mokhtar S. Bazaraa, Hanif D. Sherali, C. M.) particularly the Theorem $7.3.5$. I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Theorem $7.3.5$: Let $f:\mathbb{R}^n\to\mathbb{R}$ be differentiable, and consider the problem to minimize $f(x)$ subject to $x\in\mathbb{R}^n$. Consider an algorithm whose map $\bf{A}$ is defined as follows. The vector $y\in {\bf{A}}(x)$ means that $y$ is obtained by minimizing $f$ sequentially along the directions $d_1,d_2,\ldots,d_n$ starting from $x$. Here, the search directions $d_1,d_2,\ldots,d_n$ may depend on $x$, and each has norm $1$. Suppose that the following properties are true:

i.) There exists an $\varepsilon>0$ such that $det[D(x)]\geq\varepsilon$ for each $x\in\mathbb{R}^n$. Here $D(x)$ is the $n\times n$ matrix whose columns are the search directions generated by the algorithm, and $det[D(x)]$ denotes the determinant of $D(x)$.

ii.) The minimum of $f$ along any line in $\mathbb{R}^n$ is unique.

Given a starting point $x_1$, suppose that the algorithm generates the sequence $\{x_k\}_{k\geq 1}$ as follows. If $\nabla f(x_k)=0$, then the algorithm stops with $x_k$; otherwise, $x_{k+l}\in{\bf{A}}(x_k)$, $k$ is replaced by $k+1$, and the process is repeated. If the sequence $\{x_k\}_{k\geq 1}$ is contained in a compact subset of $\mathbb{R}^n$, then each accumulation point $x$ of the sequence $\{x_k\}_{k\geq 1}$ must satisfy $\nabla f(x)=0$.

• The theorem generates a succession $\{x_{k_j}\}_{j\geq 1}$ such that $$x_{k_j}\to x$$ and $\nabla f(x)=0$. You can see that the previous theorem is a more general version that the Gradient Descent Algorithm, For this theorem, I don't know how to take the direction vectors.

I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Thanks

Good afternoon, I am studying the book Nonlinear Programming: Theory and Algorithms (by Mokhtar S. Bazaraa, Hanif D. Sherali, C. M.) particularly the Theorem $7.3.5$. I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Theorem $7.3.5$: Let $f:\mathbb{R}^n\to\mathbb{R}$ be differentiable, and consider the problem to minimize $f(x)$ subject to $x\in\mathbb{R}^n$. Consider an algorithm whose map $\bf{A}$ is defined as follows. The vector $y\in {\bf{A}}(x)$ means that $y$ is obtained by minimizing $f$ sequentially along the directions $d_1,d_2,\ldots,d_n$ starting from $x$. Here, the search directions $d_1,d_2,\ldots,d_n$ may depend on $x$, and each has norm $1$. Suppose that the following properties are true:

i.) There exists an $\varepsilon>0$ such that $\det[D(x)]\geq\varepsilon$ for each $x\in\mathbb{R}^n$. Here $D(x)$ is the $n\times n$ matrix whose columns are the search directions generated by the algorithm, and $\det[D(x)]$ denotes the determinant of $D(x)$.

ii.) The minimum of $f$ along any line in $\mathbb{R}^n$ is unique.

Given a starting point $x_1$, suppose that the algorithm generates the sequence $\{x_k\}_{k\geq 1}$ as follows. If $\nabla f(x_k)=0$, then the algorithm stops with $x_k$; otherwise, $x_{k+l}\in{\bf{A}}(x_k)$, $k$ is replaced by $k+1$, and the process is repeated. If the sequence $\{x_k\}_{k\geq 1}$ is contained in a compact subset of $\mathbb{R}^n$, then each accumulation point $x$ of the sequence $\{x_k\}_{k\geq 1}$ must satisfy $\nabla f(x)=0$.

• The theorem generates a succession $\{x_{k_j}\}_{j\geq 1}$ such that $$x_{k_j}\to x$$ and $\nabla f(x)=0$. You can see that the previous theorem is a more general version that the Gradient Descent Algorithm, For this theorem, I don't know how to take the direction vectors.

I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Thanks

Good afternoon, I am studying the book Nonlinear Programming: Theory and Algorithms (by Mokhtar S. Bazaraa, Hanif D. Sherali, C. M.) particularly the Theorem $7.3.5$. I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Good afternoon, I am studying the book Nonlinear Programming: Theory and Algorithms (by Mokhtar S. Bazaraa, Hanif D. Sherali, C. M.) particularly the Theorem $7.3.5$. I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Theorem $7.3.5$: Let $f:\mathbb{R}^n\to\mathbb{R}$ be differentiable, and consider the problem to minimize $f(x)$ subject to $x\in\mathbb{R}^n$. Consider an algorithm whose map $\bf{A}$ is defined as follows. The vector $y\in {\bf{A}}(x)$ means that $y$ is obtained by minimizing $f$ sequentially along the directions $d_1,d_2,\ldots,d_n$ starting from $x$. Here, the search directions $d_1,d_2,\ldots,d_n$ may depend on $x$, and each has norm $1$. Suppose that the following properties are true:

i.) There exists an $\varepsilon>0$ such that $det[D(x)]\geq\varepsilon$ for each $x\in\mathbb{R}^n$. Here $D(x)$ is the $n\times n$ matrix whose columns are the search directions generated by the algorithm, and $det[D(x)]$ denotes the determinant of $D(x)$.

ii.) The minimum of $f$ along any line in $\mathbb{R}^n$ is unique.

Given a starting point $x_1$, suppose that the algorithm generates the sequence $\{x_k\}_{k\geq 1}$ as follows. If $\nabla f(x_k)=0$, then the algorithm stops with $x_k$; otherwise, $x_{k+l}\in{\bf{A}}(x_k)$, $k$ is replaced by $k+1$, and the process is repeated. If the sequence $\{x_k\}_{k\geq 1}$ is contained in a compact subset of $\mathbb{R}^n$, then each accumulation point $x$ of the sequence $\{x_k\}_{k\geq 1}$ must satisfy $\nabla f(x)=0$.

• The theorem generates a succession $\{x_{k_j}\}_{j\geq 1}$ such that $$x_{k_j}\to x$$ and $\nabla f(x)=0$. You can see that the previous theorem is a more general version that the Gradient Descent Algorithm, For this theorem, I don't know how to take the direction vectors.

I'm not sure I understand this theorem, Do any of you know how to apply the theorem or some methods based on that method?

I really appreciate the references you can give me.

Thanks