All Questions
Tagged with nonlinear-optimization global-optimization
77 questions
3
votes
1
answer
138
views
Handling absolute value and other discontinuities in numerical optimization methods that use gradients
Suppose we have difficult peak fitting problems where the the users wish to fit asymmetric peaks to their experimental data by the least squares method. One such function is illustrated below:
Here
$...
- 879
0
votes
0
answers
36
views
How to prove the convergence of Gechberg-Saxton algorithm?
I just have a problem that Gerchberg-Saxton algortihm is no worse than the previous iteration
but not sure whether it is convergent.
1
vote
1
answer
84
views
optimization over moving domains
Let $A, B$ be Banach spaces, and for any $a\in A$, $B_a\in B$ is a measurable subset. Consider the following optimization problem:
$$L(a)=\inf_{b\in B_a}\ell(b),$$
where $\ell(b)$ is a infinite-times ...
- 87
3
votes
1
answer
233
views
Min problem on integers
Let $n$ be any integer greater than $2^{10^6}$. Given any $s\le (\log_2 n)/1000$ integers $1=q_1\le q_2\le \cdots q_{s-1}\le q_s=n$. Prove that
$$\min_\ell\left(\sum_{i=1}^\ell q_i\right)\left(\sum_{i=...
0
votes
0
answers
55
views
Relationship of optimal solutions between the total function and the sub function
This is an unconstrained convex optimization problem. Let $\mathcal{N}=\left\{1,\ldots,n\right\}$, $2\leq n<\infty$. Suppose there are many strongly convex functions $f_i(x)$, where $x\in\mathbb{R}^...
- 1
0
votes
2
answers
531
views
Any idea of solving an optimization problem with cubic constraints?
I have the following optimization problem with cubic constraints, which is hard to solve. Are there any ideas, or related references, of solving such a problem?
$$ \begin{array}{ll} \underset {y, z} {\...
- 21
1
vote
2
answers
121
views
How to solve the optimization problem $\max_{\mathbf{w}}\sum_i\text{sign}(\mathbf{w}^T \mathbf{x}_i)$?
I am looking for an algorithm to solve the following optimization problem
$$\max_{\mathbf{w}}\sum_i\text{sign}(\mathbf{w}^T \mathbf{x}_i)$$
where $\mathbf{w}$ and each $\mathbf{x}_i\in\mathbb{R}^d$.
...
- 175
1
vote
0
answers
97
views
How to solve the following optimization problem?
Let $G=(V,E)$ be a connected network with $|V|=n$. Consider the following optimization problem
I'm trying to know under which conditions the following minimization problem has solution :
$${\sum _{i=1}...
- 47
2
votes
0
answers
44
views
Convergent algorithm for minimizing nonconvex smooth function
Let $\Phi$ be the Gaussian CDF and for $\gamma\ge 0$ and $h>0$, define a loss function $\ell_h:\{\pm 1\} \times \mathbb R$ by
$$
\ell_{\gamma,h}(y,y') := \phi_{\gamma,h}(yy') := \Phi((yy'-\gamma)/h)...
- 6,853
0
votes
0
answers
222
views
Convergence of ODE solutions almost everywhere to a stable equilibrium point
Theorem: Suppose ${\bf g} :\mathbb{R}^n \mapsto \mathbb{R}^n$ is continuously differentiable, there exists a set $\mathcal{A} \subset \mathbb{R}^n$ such that $\bf g$ is uniformly Lipschitz on $\...
- 1
5
votes
2
answers
248
views
Integrals can sometimes be computed through their saddle points. Are there examples of converse, when saddle points are found via integrals?
Under some reasonable assumptions integrals with large exponents can often be computed via saddle point approximations, e.g.
$$\int e^{-\lambda f(x)}\approx e^{-\lambda f(x_0)},\qquad \lambda\to\infty$...
- 319
1
vote
1
answer
190
views
Proof of extended version of non-random "almost supermartingale"
In this question, a non-random version of "almost supermartingale" theorem is proved.
Here, I would like to extend/apply the non-random version to the slightly different situation. I wonder ...
- 45
1
vote
1
answer
271
views
Can we invoke "almost supermartingale" Theorem for deterministic sequences?
Perhaps stupid question.
Question: Can "almost supermartingale" theorem be equally applicable to prove the convergence of some algorithms solving non-random optimization problems?
Attempt ...
- 45
3
votes
0
answers
91
views
What is the name for this type of optimization problem?
As we all know, a classic optimization problem can be represented in the following way:
Given a function $f: A \to \mathbb{R}$, find an element $x_0 \in A$ such that $f(x_0) \le f(x)$ for all $x \in ...
- 141
0
votes
1
answer
143
views
Transformation of an unconstrained binary quadratic optimization problem into a constrained binary linear programming problem
I know that a constrained linear optimization problem can be transformed into an unconstrained binary quadratic optimization problem (UBQP). Does anyone know if the inverse result is solved in the ...
- 61
0
votes
0
answers
92
views
Optimization problem where the objective function returns a function instead of a real number
As we all know, a classic optimization problem can be represented in the following way:
Given: a function $f: A \rightarrow \mathbb{R}$ from some set $A$ to the real numbers
Sought: an element $x_0 ∈ ...
- 141
2
votes
0
answers
47
views
Why not use global optimization algorithms like PSO to solve decentralized control problems?
I do not see many works that use global optimization algorithms to solve decentralized control problems. Here the decentralized control problem means some entries of the feedback matrix are ...
- 21
1
vote
0
answers
54
views
Nested, successive minimization solved by asympotic minimization?
I am curious about the general relation between nested, successive minimization (M1) and asymptotic minimization (M2) as defined in the following. What one wants is to implicitly minimize a sequence ...
- 390
3
votes
0
answers
282
views
Continuum of Lagrange multipliers, duality gap, and minimax theorem
Suppose I have a linear optimization problem involving random variables on some (infinite) probability space $\Omega$. For example, need to maximize expectation $E[Q]$ of random variable $Q$ subject ...
- 781
4
votes
1
answer
90
views
Do Pareto critical points of a multicriteria optimization problem form an attractor of the dynamical system induced by a descent algorithm?
Let $d\in\mathbb N$, $k\in\mathbb N$ and $f:\mathbb R^d\to\mathbb R^k$ be differentiable. Say that $v\in\mathbb R^d$ is a descent direction at $x\in\mathbb R^d$ if ${\rm D}f(x)v<0$ (component-wise) ...
- 167
0
votes
1
answer
146
views
What to call a function that is negative on a set
Let $Y$ be a nonempty region in $\mathbb{R}^n$. I am designing an algorithm which given a point $x_0$ outside $Y$ in a finite number of steps lead to a point $x_n∈ Y$. The way I do it is that I have a ...
- 679
5
votes
1
answer
242
views
Minimizing the largest eigenvalue of matrix product
Let $A\in\mathbb{C}^{m\times n}$, $B\in\mathbb{C}^{n\times k}$, $C\in\mathbb{C}^{k\times m}$ be given complex matrices. The objective of the optimization problem is
\begin{equation}
\mathop {\arg \min ...
- 377
0
votes
1
answer
329
views
Gradient-descent "type" Methods for non-convex and non-smooth functions
Most (stochastic) "gradient descent" type algorithms (such as Nesterov-accelerated gradient-descent or ADAM) seem to be well-defined only for functions which are either:
lower semi-...
- 5,405
2
votes
0
answers
49
views
A question about strong slopes (nonsmooth analysis)
Context. I'm reading the manuscrip "Nonlinear Error Bounds via a Change of Function" by Dominique Azé and Jean-Noël Corvellec (J Optim Theory Appl 2016), and I'm having a hard time ...
- 6,853
0
votes
0
answers
36
views
Optimizing upper and lower bounds
Let $L_i:X\rightarrow [0,\infty)$ be continuous (objective) functions defined on a metric space $X$ and suppose that
$$
L_1(x)\leq L_2(x)\leq L_3(x)\qquad (\forall x \in X).
$$
Here, I imagine that $...
- 5,405
3
votes
1
answer
261
views
When is the optimum of an optimization problem affine in the constraint parameter?
While working on a variational problem I have reached to the following question:
Let $f:(0,\infty) \to [0,\infty)$ be a $C^1$ function satisfying $f(1)=0$. Suppose that $f(x)$ is strictly increasing ...
- 6,741
1
vote
1
answer
195
views
Is the minimum of a constraint optimization problem differentiable in the constraint parameter?
Let $h:\mathbb R^{>0}\to \mathbb R^{\ge 0}$ be a smooth function, satisfying $h(1)=0$, and suppose that $h(x)$ is strictly increasing on $[1,\infty)$, and strictly decreasing on $(0,1]$.
Let $s&...
- 6,741
2
votes
0
answers
406
views
Pros and cons of using integer programming alone or combined integer and global optimization?
First, I am not sure if this is the right question to ask in this forum. But I have been looking for answers for a long time and I have been also asking my university's "engineering" professors but I ...
0
votes
0
answers
43
views
Minimizing along independent directions, nonlinear programming
Good afternoon, I am studying the book Nonlinear Programming: Theory and Algorithms (by Mokhtar S. Bazaraa, Hanif D. Sherali, C. M.) particularly the Theorem $7.3.5$. I'm not sure I understand this ...
- 193
0
votes
0
answers
68
views
Numerically solve a specific saddle-point problem
Let $(\Omega,\mathcal E,\mu)$ be a probability space, $k\in\mathbb N$, $$W:=\left\{w:E\to[0,\infty)^k:\sum_{i=1}^kw_i=1\;\mu\text{-almost surely}\right\},$$ $G$ be a finite nonempty set and $a^{(g)}:E\...
- 167
2
votes
0
answers
131
views
Can we conclude $\sup_g\int f_1g\le\sup_g\int f_2g$ from $\int f_1\le\int f_2$ in this situation?
Disclaimer: Please bear with me, the question isn't as complicated as it looks like, but I wasn't able to find any simplification for which no counterexample comes to my find.
Let $(E,\mathcal E,\...
- 167
0
votes
0
answers
95
views
How to maximum L1 norm problem?
I have met a problem these days.
\begin{equation}
\underset{\omega}{\max} \quad \Vert \text{diag}(\mathbf{h}^H)\mathbf{G}^H\mathbf{\omega}\Vert_1 \\
s.t.\quad\mathbf{\omega}^H\mathbf{G}\mathbf{G}^H\...
- 171
0
votes
0
answers
44
views
Is there a multiplier rule for this minimization problem?
Let $(E,\mathcal E)$ be a measurable space, $W\subseteq\left\{w:E\to\mathbb R\mid w\text{ is }\mathcal E\text{-measurable}\right\}$ be a Banach space, $k\in\mathbb N$ and $f:W^k\to[0,\infty)$. I'm ...
- 167
1
vote
1
answer
189
views
Fritz-John conditions: Equality-constrained case as special case of inequality constraints
In Chapter 4 of Nonlinear Programming: Theory and Algorithms by Bazarra, Sherali, and Shetty, the following claim is made after Theorem 4.3.2 (Fritz-John necessary conditions):
"Note also that these ...
- 11
0
votes
0
answers
101
views
How can we analytically solve this max-sum-min problem?
Let $I$ be a finite set, and $A_{ij},B_{ij},x_i,y_j\ge0$. I want to find the choice of $x_i,y_j$ maximizing $$\sum_{i\in I}\sum_{j\in J}A_{ij}\min\left(x_i,B_{ij}y_j\right)\tag1$$ subject to $$\sum_{i\...
- 167
1
vote
1
answer
232
views
Maximize a Lebesgue integral subject to an equality constraint
I want to maximize $$\Phi_g(w):=\sum_{i\in I}\sum_{j\in I}\int\lambda({\rm d}x)\int\lambda({\rm d}y)\left(w_i(x)p(x)q_j(y)\wedge w_j(y)p(y)q_i(x)\right)\sigma_{ij}(x,y)|g(x)-g(y)|^2$$ over all choices ...
- 167
8
votes
5
answers
2k
views
Prove that this expression is greater than 1/2
Let $0<x < y < 1$ be given. Prove
$$4x^{2}+4y^{2}-4xy-4y+1 + \frac{4}{\pi^2}\Big[
\sin^{2}(\pi x)+ \sin^{2}(\pi y) + \sin^{2}[\pi(y-x)] \Big] \geq \frac{1}{2}$$
I have been working on this ...
- 99
5
votes
1
answer
262
views
An effective way for the minimization of $\left\|ABA^{-1}-C\right\|$
Supposing I have complex square matrices $B_i$ and $C_i$ ($i = 1,\dots,N$) of dimension $4 \times 4$.
Is there an effective algorithm for solving the following problem?
$$\begin{align}
A=&\...
- 171
1
vote
1
answer
124
views
Finding $P$ points among $N$ to approximate a probability density function?
Let $f$ be a probability density function (positive such that $\int_{\mathbb{R}} f(x) \mathrm{d} x = 1$) and $X_0 = \{x_n\}_{1\leq n \leq N}$ be $N$ given real points. We also fix $1 \leq P \leq N$ ...
- 2,306
3
votes
3
answers
550
views
Looking for a very particular kind of non-convex functions
I want some examples (hopefully parametric families!) of non-convex functions which satisfy the following properties simultaneously,
It should be at least twice differentiable.
It should have a ...
- 2,246
1
vote
0
answers
267
views
Minimum Preserving Transformations [closed]
If $f:X\rightarrow Y$, $g:Y\rightarrow Y$ are functions and $g$ is monotone increasing function then
$$
\operatorname{argmin}_{x \in X} f(x)
=
\operatorname{argmin}_{x \in X} g\circ f(x) .
$$
X and Y ...
- 5,405
1
vote
2
answers
603
views
Maximizing a sum of Gaussians
Let $\mathbf{x}_1, \dots, \mathbf{x}_n \in \mathbb{R}^d$ be $n$ given vectors. Define the function
$$ \mathcal{K}(\mathbf{x},\mathbf{y}) := \alpha\exp\left(-\frac{\|\mathbf{x}-\mathbf{y}\|^2}{2\sigma^...
- 1,421
4
votes
2
answers
6k
views
Maximizing a convex function with a convex constraint
Given a convex function $f : \mathbb{R}^n \to [0,\infty)$, the objective is to find the farthest point in the level set $\left\lbrace x \in \mathbb{R}^n \mid f(x) \leq 1\right\rbrace$ (Assuming that ...
- 151
0
votes
1
answer
86
views
Linearly constrained saddle-point optimization
Let $f(x,y)$ be a smooth (twice differentiable) saddle function (convex in $x$ and concave in $y$), where $f \colon X \times Y \rightarrow \mathbb{R}$, and $X \subset \mathbb R^n$, $Y \subset \mathbb ...
- 884
12
votes
2
answers
350
views
A (reverse)-Minkowski type inequality for symmetric sums
Let $(u_1, u_2, u_3, u_4)$ and $(v_1, v_2, v_3, v_4)$ be vectors in $\mathbb R_+^4$. Is the following inequality true?
\begin{align*}
\left(\sum_{{[4] \choose 3}} \sqrt{u_i u_j u_k}\right)^{2/3} + \...
- 1,034
1
vote
0
answers
81
views
Maximizing sum of homogeneous functions of order one over a polytope
Let $f_i: \mathbb{R}^n\rightarrow \mathbb{R}$ be
concave, increasing (i.e., if $x\geq y$ where the inequality is entry wise, we have $f_i(x)\geq f_i(y)$), and a
homogeneous function of order one for ...
- 393
1
vote
0
answers
190
views
linear relaxation of an optimization problem
I'm reading an article lately, and there is one point which confuses me.
So, we have the following constrained binary quadratic problem.
min $x^{T}Qx$
with the constraints that $Ax\leq b$ and $x\in ...
- 11
2
votes
1
answer
437
views
equality between the ratio trace and the determinant ratio
I have encountered the following equality
$\arg\max_{\text{tr}\left(\boldsymbol{S}^{H}\boldsymbol{S}\right)=1}\text{tr}\left(\boldsymbol{S}^{H}\boldsymbol{A}\boldsymbol{S}\left(\boldsymbol{S}^{H}\...
- 503
4
votes
3
answers
200
views
Maximizing a pseudoconcave function in a box
I am trying to solve the problem:
$\max_{\boldsymbol{s}\in\mathbb{R}^{n}} \frac{\sqrt{\boldsymbol{a}^{T}\boldsymbol{s}+\alpha}}{\boldsymbol{b}^{T}\boldsymbol{s}+\beta}\\
\text{s.t} \;\;0\leq s_{i}\...
- 503
3
votes
0
answers
239
views
Constrained optimization with a Proportional-Integral-Derivative (PID) controller
My engineering colleagues have devised an interesting approach to equality-constrained optimization. I.e. they wish to solve the problem $\min_x f(x)$ subject to the constraint $g(x) = 0$ where $f, g ...
- 163