Questions tagged [convex-optimization]
Optimization with convex constraints and convex objectives; notions related to convex optimization such as sub-gradients, normal cones, separating hyperplanes
359 questions with no upvoted or accepted answers
13
votes
0
answers
710
views
Minimizing total variation under constraint
For $p\in[0,1]$, we write $\mathrm{Ber}(p)$
to denote the Bernoulli measure on $\{0,1\}$;
that is, $\mathrm{Ber}(p)(0)=1-p$,
$\mathrm{Ber}(p)(1)=p$.
For $n\in\mathbb{N}$ and $p=(p_1,\ldots,p_n)\in[0,1]...
- 6,541
9
votes
0
answers
1k
views
Existence of barycenter
Let $(X,d)$ be a metric space. A barycenter of a Borel probability measure $\mu$ on $X$ is a minimizer of the function
\begin{equation}
\begin{split}
f \colon X & \to \mathbb{R}\\
x &\mapsto \...
- 1,347
8
votes
0
answers
459
views
Are there any characterizations of $C^2$ convex functions?
There are several characterizations of convex functions with the Lipschitz continuous gradient. If we already know that the function is of class $C^1$, then we have the following equivalent conditions:...
- 28k
8
votes
0
answers
210
views
Concavity of product and ratio of sums
Apologies if this question is not appropriate for MathOverflow. I have asked at Math.StackExchange without success.
Consider the function $f:\mathbb{R}^n\rightarrow \mathbb{R}$ defined as
$$
f(x)=\...
- 379
8
votes
0
answers
254
views
Quantum coupon collection: positivity of an alternating sum of matrices
It is well-known that in the classic coupon collecting problem (CCP), the expected waiting time is
\begin{equation*}
T_n(x_1,\ldots,x_n) = \sum_{k=1}^n (-1)^{k+1}\sum_{1\le i_1 < \cdots < i_k \...
- 28.6k
7
votes
0
answers
249
views
Proving this function is convex
Let $C$ be a symmetric positive definite matrix such that $0\leq c_{ij} \leq 1$, $c_{ii}=1$, and define $f$ as $$f(x)=\sum_{i}x_{i}\log(\sum_{j}c_{ij}x_{j})$$ for positive vectors $x$ (in fact let's ...
- 4,049
7
votes
0
answers
1k
views
Closed-form solution of a linear programming question
Among all the probability matrices
\begin{equation*}
P =
\left(\begin{array}{cccc}
p_{00} & p_{01} & \ldots & p_{0,J-1} \\
p_{10} & p_{11} & \ldots & p_{1,J-1} \\
\vdots & \...
7
votes
0
answers
217
views
Characterizing matrices with rank constraint
Given matrix $M\in\Bbb\{0,1\}^{n\times n}$, is there a nice method to characterize $$\mathscr{D}[M]=\{Q\in\Bbb\{0,1\}^{n\times n}:\mathsf{rank}(M-Q)= \mathsf{rank}(Q),\quad M-Q\in\Bbb\{0,1\}^{n\times ...
- 13.9k
7
votes
0
answers
209
views
Numerical linear algebra: how to compute $B^TC^{−1}B$ efficiently
Hi,
my question is similar to this one. I have to compute $B^TC^{−1}B$, where $C$ is a strictly positive definite $n\times n$ matrix and $B$ is $n\times m$.
The matrix $C$ is huge ($n$ up to a ...
- 619
6
votes
0
answers
48
views
Strengthening the Kovner-Besicovich theorem: Does every unit-area convex set in the plane contain a centrally symmetric hexagon of area $2/3$?
The Kovner-Besicovich theorem states that every convex set $S$ in the plane contains a centrally symmetric subset $C$ of at least $2/3$ the area of $S$, and that this bound is sharp for triangular $S$....
- 3,068
6
votes
0
answers
136
views
Minimizing $\det(D)$ for all diagonal matrices $D$ that satisfy $D+B \succeq 0$
Let $A$ be an $n \times n$ real matrix and let $B$ be the block bipartite matrix
$$B = \begin{bmatrix} 0&A \\
A^{T}&0 \end{bmatrix}$$
I came across the following optimization problem, which ...
- 401
6
votes
0
answers
255
views
Concavity of a function implicitly defined by a polynomial
Consider the following system of $n$ equations:
\begin{equation}f_j^2 = x_j^2\sum_{i=1}^n A_{ij} f_i
\tag{$\star$}
\end{equation}
where $A_{ij}\geq 0$ are known constants and where $x_j>0$ for ...
- 379
6
votes
0
answers
97
views
Finding the optimal mixture of two convex functions
I am trying to find an efficient way to solve the problem $$\min_{p,x_1,x_2} p\cdot f(x_1)+ (1-p) \cdot f(x_2)~~~~~ s.t.\\p\cdot g_1(x_1) + (1-p)\cdot g_2(x_2)\leq 1 \\ 0\leq p \leq 1$$ where $x_1,x_2\...
5
votes
0
answers
259
views
Does the refined Slater's condition hold also in the infinite-dimensional case?
Let $X$ be an infinite-dimensional Banach space. I have the following optimization problem.
$$\begin{array}{ll} \underset{x \in X}{\text{minimize}} & f(x)\\ \text{subject to} & g_1(x) \leq 0\\ ...
- 51
5
votes
0
answers
95
views
Differentiability/continuity of stabilizing solution to algebraic Riccati equation with respect to matrix parameters
When solving the LQR problem to find the optimal feedback matrix $K^*$, i.e. solving
\begin{align*}
\min_K &\int_0^\infty \left(x^TQx + u^TRu\right)dt,\\
\text{s.t. }&\dot{x} = Ax+Bu,\\
&u=...
- 151
5
votes
0
answers
252
views
Generalized convexity
Let $X$ be a vector space. The positive-homogeneous function $\|\cdot\|$ is said to be a quasinorm
if $\|x+y\|\le K(\|x\|+\|y\|)$, for some $K\ge1$; it is a norm if $K=1$.
Question: 1. (terminology) ...
- 6,541
5
votes
0
answers
135
views
Conv A = Dual B
I have two cones $A$ and $B$ in a Euclidean space.
I want to show that $\mathrm{Conv}\,A=\mathrm{Dual}\,B$; that is, $a\in\mathrm{Conv}\,A$ if and only if $\langle a,b\rangle\ge 0$ for any $b\in B$.
...
- 45k
5
votes
0
answers
147
views
Projection of a function $f\in L^1(\Omega)$ onto a finite dimensional subspace
Suppose $\Omega \subset \Bbb R^n$ be a domain such that $|\Omega|<\infty$, $f\in L^1(\Omega)$. Let $Y= \text{span}\{g_1,\dots, g_k\}$.
Is there a characterization of the set of projections of $f$...
- 1,245
5
votes
0
answers
269
views
Fenchel conjugate on a Hadamard manifold
Let $M$ be a Hadamard manifold and let $F:M\to\mathbb{R}$ be a real-valued convex function on $M$. What would be the Fenchel-Young conjugate of $F$?
In general for a real locally convex vector space $...
- 364
5
votes
0
answers
151
views
Dimensions of faces of convex hull of convex bodies
Let $K_1,\ldots,K_m\subset\mathbb{R}^n$ with $m\geq n$ some convex bodies (i.e. compact with nonempty interior). I am interested in sufficient criteria for the convex hull $K=\textrm{conv}(K_1,\ldots,...
- 3,031
5
votes
0
answers
330
views
Best Approximation in Operator/non-Frobenius Norm
Since the Frobenius norm on matrices is generated by an inner product, solving the optimization/approximation problem of approximating an operator $X$ with a scalar multiple of another operator $Y$
$$\...
- 153
5
votes
0
answers
96
views
Non-convex optimization problem involving minimum spanning trees
Suppose I am given $n$ points $p_1,\dots,p_n\in \mathbb{R}^2$, as well as two positive coefficients $a_1$ and $a_2$, and I am trying to select $n$ points $x_1,\dots,x_n$ to solve the following ...
- 4,049
5
votes
0
answers
167
views
A specific case of the $p$-center problem
Given a fixed positive integer $m$, let $\cal{S}$ be the subset from $\mathbb{R}^m$ defined as $\cal{S} = \{u \in \mathbb{R}^m \mid \forall i \in \{1, \dots, m\}, u(i) > 0$ and $\sum_{i=1}^m{u(i) = ...
- 125
5
votes
0
answers
1k
views
Coordinate mirror descent
Let $f$ be a jointly convex function of 2 variables say $x,y$. I am interested in solving the optimization problem $$\min_{x,y\in\Delta} f(x,y)$$ where $\Delta$ is a $d$ dimensional simplex. An ...
- 323
4
votes
0
answers
255
views
Economic equilibrium and tropical geometry
There is a famous saying in economics: When everyone pursues his or her own interests, there is an invisible hand that brings the market to equilibrium. However, this is not always the case. Here is ...
- 123
4
votes
0
answers
622
views
Simple constructive proof for the hyperplane separating theorem (HST)?
The Hyperplane Separation Theorem (HST) is usually proved through the existence of a unique minimum-norm vector in a nonempty closed convex set. I think this is an existential proof which applies to ...
- 599
4
votes
0
answers
140
views
A convex function is "usually" subdifferentiable
Let $X$ be a locally convex topological vector space, and let $f:X\to\mathbb R\cup\{\infty\}$ be a proper, convex, lower semicontinuous function, whose effective domain $D:=f^{-1}(\mathbb R)$ is ...
- 537
4
votes
0
answers
260
views
$L^2$-projection onto monotone functions
Let $f:{\mathbb R}\rightarrow{\mathbb R}$ be measurable and such that
$$\int_{-\infty}^0(f(x)-a)^2dx+\int_0^{+\infty}(f(x)-b)^2dx<+\infty.$$
This, denoted as $E(a,b)$, is an affine space: $E(a,b):=...
- 52.3k
4
votes
0
answers
236
views
Fréchet subdifferentiation on riemannian manifolds
Context. I'm looking for a "natural" definition of subdifferentials on riemannian manifolds.
Given a function $F:\mathbb R^m \to \mathbb R$, its Fréchet-subdifferential at a point $w \in \...
- 6,853
4
votes
0
answers
229
views
How to find the dimension of the polar cone of a convex cone generated by some given vectors
Suppose we have access to a generating set $\{v_1, ..., v_k\}\subseteq\mathbb{R}^n$ of the convex cone $C=cone(v_1, ..., v_k)$, where $cone(\cdot)$ is the conical hull (i.e. nonnegative span) of ...
- 461
4
votes
0
answers
252
views
Can this function be minimized?
Let $X$ be a locally convex TVS, and let $A$ and $B$ be convex and compact subsets of $X$ with $A \subset B$.
Let $f: A \times B \to [0,\infty]$ have the following properties:
(1) For all $b \in B$, $...
- 869
4
votes
0
answers
509
views
analytic approximations of the min and max operators
Question:
What is the state of the art on analytic approximations of $\min$ and $\max$? My hunch is that numerical analysts probably have a better solution than the one I propose here.
For any $\...
- 3,871
4
votes
0
answers
228
views
Minimization over a convex function of equal vs unequal success probabilities of Bernoulli random variables
Let $U_1,U_2,\ldots,U_n$ be $n\geq 2$ mutually independent Bernoulli random variables. There are two cases of interest:
$1.$ The random variables $U_1,U_2,\ldots,U_n$ are identically distributed;
$...
- 359
4
votes
0
answers
539
views
Using Linear Programming as an iterative procedure
Suppose, we have a linear program and an optimal solution to it. Suppose now, we get a new constraint. We want to obtain an optimal solution to the given linear program extended by that new constraint....
- 391
4
votes
0
answers
141
views
Asymptotic behavior of gradient descent on a smooth, convex, non-negative function with no finite minimum - part II
Let $f: \mathbb{R}^d\rightarrow\mathbb{R}_{\geq 0}$ be a function which is convex and smooth (i.e., in $C^{\infty}$). If $x^* \in \mathbb{R}^d$ is the (global) minimum of $f$, it is well known that ...
- 968
4
votes
0
answers
307
views
Derivative of rank $r$ approximation of matrix
Let $Y \in \mathbb R^{n \times c}$ and $r$ be an integer with $1 \le r \le \operatorname{rank}(Y)$. Consider the problem
$$\text{minimize} \|Y-X\|_{\text{Fro}}^2\text{ over }X \in \mathbb R^{n \times ...
- 6,853
4
votes
0
answers
241
views
Stochastic subgradient descent almost sure convergence
I was reading up on stochastic subgradient descent, and most sources i could find via google search give quick proofs on convergence in expectation and probability, and say that proofs of almost sure ...
- 141
4
votes
0
answers
76
views
How well does an estimator perform on another dataset?
Suppose $X \sim N(0, \Sigma)$ is a $d$-dimensional Gaussian random vector. And we have $2n$ $i.i.d$ sample $X_1, \ldots, X_{n}, \ldots, X_{2n}$.
Let $\hat{\Sigma}_1 = \frac{1}{n}\sum_{i=1}^nX_i X_i^\...
- 515
4
votes
0
answers
137
views
Covering Number of a Positive Semidefinite Cone (Approximate the Objective of a SDP)
I was wondering what the covering number of a positive semidefinite cone is. Consider the semidefinite optimization program
\begin{align}
\max\langle \mathbf{C}, \mathbf{X} \rangle~~\text{subject to}~...
- 1,127
4
votes
0
answers
108
views
Designing Character Other Than Temperature for Simulated Annealing on Combinatorial Optimization
Many research on designing temperature for simulated annealing is carried out. We wonder if there is any research on designing general feature of the Hamiltonian used in Simulated Annealing.
For ...
- 909
4
votes
0
answers
205
views
Basin of Attraction
I have a function $F$ which is defined as follows:
$$
F(x) = \sum_{i=1}^N f(z_i^T x)
$$
where ${z_i}$ are known $m \times 1$ vectors, $x$ is an $m \times 1$ vector, and for $t\in \mathbb{R}$, $f(t) = \...
- 291
4
votes
0
answers
126
views
An inequality from the "Interlacing-1" paper
This question is in reference to this paper, http://annals.math.princeton.edu/wp-content/uploads/annals-v182-n1-p07-p.pdf (or its arxiv version, http://arxiv.org/abs/1304.4132)
For the argument to ...
4
votes
0
answers
184
views
This function looks quasiconvex, can't understand why
Suppose that $\mathbf{C}$ is a given matrix with non-negative entries in $\mathbb{R}^{m\times n}$ and $d$ is a given scalar, and let $g(\mathbf{y})$ be defined by $$g(\mathbf{y}):=\max_{\mathbf{x}\in\...
- 41
4
votes
0
answers
293
views
When is the sum of a weak-$*$ closed convex cone and a subspace also weak-$*$ closed?
Let $X$ be a Banach space. Suppose $C \subset X^*$ is a convex cone and $V \subset X^*$ is a subspace, and suppose both $C$ and $V$ are closed in the weak-$*$ topology. Are there any general ...
- 195
3
votes
0
answers
281
views
Interchange limit and supremum of functionals over a bounded convex set
Let $(H, \langle\cdot,\cdot\rangle)$ be a separable real Hilbert space and $B\subset H$ be (nonempty) convex and bounded, and suppose that $(\alpha_k)\subset H$ is a sequence for which the limit $\...
- 463
3
votes
0
answers
87
views
Instances of c-concavity outside of optimal transport?
Let $X$ and $Y$ be metric spaces, and let $c:X\times Y\rightarrow \mathbb{R}$ be a nonnegative function which we refer to as a cost. For any $\phi:X\rightarrow \mathbb{R}$ and $\psi:Y\rightarrow \...
- 232
3
votes
0
answers
109
views
Convex minorants to convex functions, given partial Taylor expansion and smoothness estimate
Let $V$ denote a strictly convex function (in arbitrary dimension) whose Hessian is $L$-Lipschitz. Given only this knowledge, and the values of $\left\{ V \left( x \right), \nabla V \left( x \right), \...
- 801
3
votes
0
answers
606
views
Proving an optimization problem from continuous input to binary is optimal
Suppose we have a function $f(x,y,z)$ where the inputs are uniform from 0 to 1. The output is either $+1$ or $-1$. And there is a partial symmetry $f(x,y,z) = f(z,y,x)$.
Tell me what the minimum of ...
3
votes
0
answers
91
views
What is the name for this type of optimization problem?
As we all know, a classic optimization problem can be represented in the following way:
Given a function $f: A \to \mathbb{R}$, find an element $x_0 \in A$ such that $f(x_0) \le f(x)$ for all $x \in ...
- 141
3
votes
0
answers
97
views
Projection onto level set of convex functional
Fix a probability space $(\Omega,\mathcal{F},\mathbb{P})$ and let $F:L^2_{\mathbb{P}}(\mathcal{F})\rightarrow (-\infty,\infty]$ be bounded-blow, convex, lower semi-continuous, and not identically ...
- 5,405