Questions tagged [convex-optimization]
Optimization with convex constraints and convex objectives; notions related to convex optimization such as sub-gradients, normal cones, separating hyperplanes
839 questions
2
votes
1
answer
70
views
Optimize a function with not-full knowledge of gradient
I want to optimize the following function:
$$
argmin_{x} f(x) = g(x) + h(x)
$$
, where I can get $\nabla_xg(x)$, but cannot calculate $\nabla_xh(x)$.
The derivative-free method, such as the Hill ...
- 12.7k
0
votes
1
answer
101
views
Estimation via projecting onto a convex body
Assume that $\theta$ is in a convex body $K \in \mathbb{R}^n$ and we observe $y = \theta + z$, where $z$ is a noise term (following, say, the normal distribution). Consider an estimator of $\theta$ by ...
- 128k
1
vote
0
answers
73
views
Principal component analysis with boundedness constraints
Let $A$ be an $m\times n$ matrix with entries in $F$ ($F=\mathbb{R}$ or $F=\mathbb{C}$).
It is well-known that $A$ has decompositions of the form
$$\displaystyle A = \sum_{k=1}^r\lambda_k\hspace{2mm} ...
- 2,605
0
votes
0
answers
143
views
Minimax problem : uniqueness of a solution
Let $n\geq2$. Is it true that the minimax problem:
$$
\min_{p\in\mathcal{P}}\max_{H\in\mathcal H}p^tH\bar{p},
$$
where
$\mathcal H\subset\mathcal{M}(n)$ is a strictly convex bounded subset of ...
- 4,034
1
vote
2
answers
416
views
Lower bound for $ \sum_{i=1}^n x_i f(x_i)$ when $\sum_{i=1}^{n}x_i = K$
Considering,
the set of all n dim. vectors $\{x_i\}_{i=1,...,n} $ such that $x_i \geq 0 $ and $\sum_{i=1}^{n}x_i = K$
Any continuous and strictly increasing function $f^+(x)$ : $ \mathbb R^+ \to \...
- 18.7k
0
votes
0
answers
173
views
Is $(K^*)^{**}=(K^{**})^*$ for any cone $K$?
I'm considering the dual cone $K^*$ of a non-convex cone $K$. I came up with a theory that $K^{**}$ is the closure of convex hull of $K$. Then I wonder whether $(K^*)^{**}=(K^{**})^*$ holds for any ...
- 12.7k
2
votes
2
answers
104
views
Inequality for matrix with rows summing to 1
Let $A$ be real matrix with $M > 1$ rows and $K > 2$ columns, and each entry $a_{m,k} \in (0,1)$, with each row summing to $1$. For all $m$
$$
\sum_{k=1}^{K} a_{m,k} = 1
$$
I want to find out if ...
- 20.2k
3
votes
1
answer
179
views
On the convexity of certain set of random vectors
Let ${\cal X}$ be the set of pairs of random variables $(X,Y)$ with finite expectations. For constant $c\in[0,1]$, define set
$$
\{(X,Y)\in{\cal X}:\exists a\geq 0, \, b\geq 0 \text{ such that } E[\...
- 128k
0
votes
0
answers
118
views
Weak derivative of projection onto probabilist's simplex
Let $\Delta_n:=\{x\in [0,1]^n:\boldsymbol{1}^{\top}x=1\}$ denote the probabilist's $n$-simplex and let $P:\mathbb{R}^n\rightarrow\Delta_n$ denote the (Euclidean) metric projection onto this simplex ...
- 5,405
1
vote
0
answers
98
views
Solution of a simple optimization problem
Let $\mathbf{U}_1$ and $\mathbf{U}_2$ be two arbitrary unitary matrices and $\mathbf{D}$ be a diagonal matrix. What is the solution of the following optimization problem?
\begin{align}
\min_{\mathbf{...
- 63.9k
0
votes
0
answers
124
views
The best unitary matrices that approximate a matrix product
Let $\mathbf{A}$ be an arbitrary $N\times N$ complex matrix. Moreover, $\mathcal{U}_1$ and $\mathcal{U}_2$ are distinct subsets of all unitary matrices. Suppose the matrices $\mathbf{U}_1$ and $\...
- 287
4
votes
1
answer
336
views
Which set of functions admits the existence of the minimizer?
Let $a,b \in \mathbb R$ and consider the functional $J$ on $X$:
$$J[u] = \int_0^1 \left( (u'(x))^2 -a)^2 + b \ln (1+ u^2(x))\right) dx$$
Providing reasons specify if the $\inf J$ over $X$ is attained ...
4
votes
1
answer
182
views
Subgradient in a predual under weak* continuity
Let $X$ be a Banach space. Suppose $f:X^*\to\mathbb R\cup\{\infty\}$ is convex, has weak*-compact effective domain, and is weak*-continuous on its effective domain. In particular, $f$ is weak*-lower ...
- 12.9k
1
vote
0
answers
108
views
Optimal bandwidth for a Gaussian filter
I have an $n \times n$ image $A$, and an $m\times m$ image $B$, where $n>m$. As the smaller image looks like a lower-resolution version of the larger one, I'm interested in the relative loss, ...
CommunityBot
- 1
1
vote
2
answers
278
views
Optimization of a integral function
I have a function $h(y,x_1,x_2,\ldots,x_n)$. It is known that the minimum value of $h$ for any $y$ is attained when $x_1 = x_n$ and $x_2 = x_3 = \cdots = x_{n-1}$. Now consider the following function
\...
- 211
0
votes
0
answers
159
views
Double summation of matrices as constraints in convex optimization in CVX
I want to implement the following optimization problem from the following paper Randomized Gossip Algorithms, Page 10 Eq 53:
\begin{align}
\text{minimize} &\qquad s\\
\text{subject to} & \...
- 63.9k
1
vote
0
answers
37
views
Sufficient condition for an $n$-tuple to be a convex conjugate
We say $(f_1,f_2,\dotsc,f_N)$ is a convex conjugate if for any $i=1,2,\dotsc,N$ and any $x_i\in\Bbb R^d$, we have:
$$f_i(x_i)=\sup\left\{\sum_{k=1}^{N}\sum_{j=k+1}^N x_k x_j - \sum_{j=1,j\neq i}^N f_j(...
- 12.9k
2
votes
1
answer
108
views
Subgradient of a convex integral
I have an integral to minimize that writes like $$F: \mathbb R^d \to \mathbb R: \theta \mapsto \int_{[0,1]^d} f(\langle x,\theta\rangle) dx$$.
The function $f$ is a convex function, which makes $F$ a ...
- 6,696
2
votes
1
answer
400
views
Smoothness of Minkowski functional is equivalent to smoothness of boundary
Let $C\subseteq \mathbb{R}^n$ be a convex body containing $0$ in its interior. I recently read that Minkowski functional of $C$,
$$
f_C(x):=\inf\Big\{t>0:\frac1{t}\cdot x\in C\Big\},
$$
is $C^1$ ...
- 63.9k
1
vote
0
answers
157
views
Is $x\cdot f(x)$ quasiconvex for $x>0$, if $f(x)$ is monotonically decreasing, convex, and positive? [closed]
Original question
Given $f(x):\mathbb{R}^+\to\mathbb{R}^+$, which is monotonically decreasing and convex. Then define a function $g(x) = xf(x)$, I am wondering whether $g(x)$ is quasiconvex for $x>...
- 41
3
votes
0
answers
606
views
Proving an optimization problem from continuous input to binary is optimal
Suppose we have a function $f(x,y,z)$ where the inputs are uniform from 0 to 1. The output is either $+1$ or $-1$. And there is a partial symmetry $f(x,y,z) = f(z,y,x)$.
Tell me what the minimum of ...
4
votes
2
answers
415
views
Selecting vertices in a convex polygon
Given $n$ vertices of a convex polygon in $\mathbb{R}^2$, selecting two points that are furthest apart is done by finding the diameter in a convex polygon. But how can one select three vertices such ...
- 1,251
4
votes
5
answers
2k
views
Reference request: importance of Lipschitz continuity
I see that Lipschitz continuity is a common assumption used in optimisation, statistics, machine learning, etc.
Could you point me in the direction of some literature that discusses why Lipschitz ...
- 12.7k
2
votes
0
answers
152
views
A truncated Frobenius norm of a matrix is convex or not?
Given a positive integer $k$ and a matrix $X\in \mathbb{R}^{m\times n}$. A truncated frobenius norm of a matrix $X$ is defined by
$$\Vert X \Vert_{k,F} = \sqrt{\sum_{i=k+1}^{m} \sigma_i^2(X)},$$
where ...
- 143
2
votes
0
answers
57
views
A variant of the elliptope relaxation
Given a p.s.d. matrix $A$, one may want to find:
$$
\max_x x^t A x \mbox{ such that } x \mbox{ has entries }1 \mbox{ or } {-1}.
$$
This hard problem has a well known relaxation based on the so called ...
3
votes
0
answers
91
views
What is the name for this type of optimization problem?
As we all know, a classic optimization problem can be represented in the following way:
Given a function $f: A \to \mathbb{R}$, find an element $x_0 \in A$ such that $f(x_0) \le f(x)$ for all $x \in ...
0
votes
0
answers
127
views
On least-squares with positive semidefinite constraints
Given real symmetric matrix $\mathbf{R} \in \mathbb{S}^{n\times n}$ and matrices $\mathbf{X}_n, \mathbf{X}_{n-1} \in \mathbb{R}^{n \times m}$,
$$\begin{array}{ll} \underset{\mathbf{A} \in \mathbb{R}^{...
0
votes
1
answer
588
views
Minimum conditional expectation of complement of event given conditional expectation of event?
Suppose $X$ is a pdf over $[0,m]$ and $Y$ is a binary experiment on $X$ such that $P(Y=1|X)$ is continuous, and we have that $\mathbb{E}[X|Y=1] = \mu_y$ and $\mathbb{E}[X] < \mu_y$. Is it always ...
CommunityBot
- 1
0
votes
0
answers
109
views
How to find a set given its support function
Let $\mathcal{U}$ be a convex and compact set. Its support function is defined as $\delta^*(v|\mathcal{U})=\sup_{u\in \mathcal{U}} v^T u$. Assume that we are given the support function $\delta^*(v|\...
- 53
2
votes
0
answers
37
views
Stochastic gradient descent in 'stronger' settings
I am minimzing a function $F(x) = \mathbb E(f(x,\Xi))$ where $\Xi$ is some random value, by a stochastic gradient descent that generates a random number $\xi$ from the distribution of $\Xi$ at each ...
- 686
3
votes
0
answers
97
views
Projection onto level set of convex functional
Fix a probability space $(\Omega,\mathcal{F},\mathbb{P})$ and let $F:L^2_{\mathbb{P}}(\mathcal{F})\rightarrow (-\infty,\infty]$ be bounded-blow, convex, lower semi-continuous, and not identically ...
- 63.9k
1
vote
0
answers
52
views
When the summands of a positive definite matrix are positive definite
Let $A,B$ be two real symmetric matrices. Let $C = A+B$ be a positive-definite matrix. Write $C>0$ for $C$ being positive-definite. Suppose that $A>0 \implies C>0$ and $B > 0 \implies C>...
- 71
3
votes
1
answer
227
views
Relaxations for the spectral norm maximization problem
Optimizing the spectral norm of some positive semidefinite matrix $A(x) \in S^{n}$, w.r.t. a list of variables $x \in \mathbb{R}^d$ and semidefinite constraints is, in general, a nonconvex problem (...
- 1,517
0
votes
0
answers
92
views
Optimization problem where the objective function returns a function instead of a real number
As we all know, a classic optimization problem can be represented in the following way:
Given: a function $f: A \rightarrow \mathbb{R}$ from some set $A$ to the real numbers
Sought: an element $x_0 ∈ ...
- 141
4
votes
1
answer
210
views
On some convergence theorems by Felix E. Browder (1967)
I have been reading Felix E. Browder's Convergence Theorems for Sequence of Nonlinear Operators in Banach Space and I was hoping I could find answers to a couple of questions I have about the paper. ...
- 6,367
3
votes
1
answer
621
views
Strong convexity inequality w.r.t. infinity norm $\lVert\cdot\rVert_{\infty}$
Consider a twice differentiable 1-strongly convex function $f:\mathbb{R}^n \to \mathbb{R}$.
Is it true that there exists $\alpha>0$ independent of $n$ such that, for all $x \in \mathbb{R}^n$:
\...
- 12.9k
1
vote
0
answers
59
views
Minimizing square roots with the consecutive ones property
Let $A=[a_{ik}]$ be a matrix with the consecutive ones property in each column, i.e. each column consists of a single consecutive block of $1$'s (with zeros everywhere else). Is there anything at all ...
- 4,049
1
vote
0
answers
140
views
Factorization of argmax
We consider a function $f(s_{1:p}, a_{1:p})$, where $p>1$ is an integer, $s_{1:p}$ denotes $(s_1,\ldots,s_p)^\top \in R^p$, and $a_{1:p}$ denotes $(a_1,\ldots,a_p)^\top \in R^p$.
Question: What is ...
- 1,127
1
vote
0
answers
58
views
Second-order envelope theorem for linear programming
Consider parameterized linear programming $V(\theta) = \max_x \langle c(\theta),x\rangle$ s.t. $A(\theta)x\leq b(\theta)$, $x\geq 0$. Let's also assume $c,A,b$ are infinitely differentiable with ...
- 373
1
vote
0
answers
98
views
Optimization with convolution in the objective function
I would like to minimize the following objective function
$$
\| H \ast A - (H \cdot I) \ast B \|_F^2
$$
w.r.t. $H$, where
$H$, $I$, $A$, and $B$ are all square matrices of the same size ($I$ is a ...
- 6,367
4
votes
1
answer
254
views
Max-norm projection of a Hermitian matrix onto the set of positive semidefinite matrices
For a given Hermitian matrix $A$ (i.e. complex matrix with $A_{ij}^{\ast}=A_{ji}$) find its max-norm projection onto the set of complex positive semi-definite matrices:
$$\Pi(A)=\mathrm{argmin}_{M\...
CommunityBot
- 1
1
vote
0
answers
62
views
Derivative of a function of ordered variables
Can I differentiate
$$(\pmb{y}^o - (\pmb{x}\cdot\pmb{a})^o)^\top(\pmb{y}^o - (\pmb{x}\cdot\pmb{a})^o)$$ with respect to $\pmb{a}$? (I want to minimize the expression with respect to $\pmb{a}$.)
Here, $...
- 133
1
vote
1
answer
211
views
Does coercivity/supercoercivity conjugates?
According to Wikipedia, a function $f: \mathbb{R}^n \to \mathbb{R} \cup \{-\infty, +\infty\}$ is called coercive if,
$$f(x) \to +\infty \text{ as } \|x\| \to +\infty$$
and it is super-coercive if
$$\...
- 12.7k
2
votes
0
answers
162
views
Three-constraint homogeneous QCQP
Consider the homogeneous quadratically constrained quadratic program,
$$\min_{u^T u =1} u^T A_1 u$$
$$\textrm{subject to}\quad u^T A_2 u = 0,\quad u^T A_3 u = 0$$
This problem is particularly studied ...
- 1,203
4
votes
2
answers
316
views
Average value of $\frac{x'A^2x}{x'A^3x}$ over surface of $n$-dimensional sphere
Suppose $A$ is a diagonal matrix with eigenvalues $1,\frac{1}{2},\frac{1}{3},\ldots,\frac{1}{n}$ and $x$ is drawn from standard Gaussian in $n$ dimensions. Define $z_n$ as follows
$$z_n=E_{x\sim \...
- 128k
1
vote
1
answer
791
views
Hardness of concave minimization problem
I have an optimization problem $\underset{x}{\min} ~ c(x) - k \cdot x$ where $c(x)$ is a non-decreasing concave function with $c(0) = 0$, $x \in C \subseteq \mathbb{R}^d_{\geq 0}$. By non-decreasing, ...
CommunityBot
- 1
0
votes
0
answers
78
views
Pareto-optimal front $F$ in $m$-dimensional space can not have more than $\mathbf{H}_{m-2}(F)$ homology groups
I need to prove that a Pareto-optimal front $F$ in $m$-dimensional space (i.e. $m > 1$) can not have more than $\mathbf{H}_{m-2}(F)$ homology groups.
What it simply means that in a 2-dimensional ...
- 101
5
votes
0
answers
95
views
Differentiability/continuity of stabilizing solution to algebraic Riccati equation with respect to matrix parameters
When solving the LQR problem to find the optimal feedback matrix $K^*$, i.e. solving
\begin{align*}
\min_K &\int_0^\infty \left(x^TQx + u^TRu\right)dt,\\
\text{s.t. }&\dot{x} = Ax+Bu,\\
&u=...
- 63.9k
0
votes
1
answer
87
views
How to find $y_u?$
In the paper Semi-supervised learning by mixed label propagation Wei Tong and Rong Jin define
$S$ as the similarity(adjacency) matrix
$D = \operatorname{diag}(D_1, D_2, \ldots, D_n)$ where $D_i = \...
- 13
2
votes
0
answers
47
views
Why not use global optimization algorithms like PSO to solve decentralized control problems?
I do not see many works that use global optimization algorithms to solve decentralized control problems. Here the decentralized control problem means some entries of the feedback matrix are ...
- 21