All Questions
Tagged with convex-optimization na.numerical-analysis
30 questions
0
votes
2
answers
97
views
Optimization algorithms for Kronecker approximation of high-dimensional covariance matrices
I'm working with a high-dimensional covariance matrix and exploring Kronecker product approximations to make it computationally manageable.
Here's the setup:
I have a graph $G$ represented by a $D\...
- 1
0
votes
0
answers
54
views
How to deal with minimizing a flat objective function
Problematic (Debiased Sinkhorn barycenter, proposed by H.Janti et al.): Let $\alpha_1, \ldots, \alpha_K \in \Delta_n$ and $\mathbf{K}=e^{-\frac{\mathrm{C}}{\varepsilon}}$. Let $\pi$ denote a sequence ...
- 121
6
votes
1
answer
234
views
Stopping criteria for damped Newton iterations with backtracking line search
Are there better criteria than the Armijo criterion for damped Newton iteration with backtracking line search, when the objective is standard self-concordant? (See Boyd and Vandenberghe.)
Let $F(x)$ ...
- 981
1
vote
1
answer
309
views
Numerical estimation of partial derivatives of convolved functions when closed forms do not exist
Summary: Some peak functions are convolutions which may not have a closed form solution. A classical example can that of a Voigt which is a convolution of a Lorentzian and a Gaussian, followed by ...
- 879
1
vote
1
answer
69
views
Characterization of the behavior of the residuals in conjugate gradient
In conjugate gradient method for solving symmetric positive definite linear system $Ax=b$, which can also be regarded as a convex optimization problem $\dfrac{1}{2} x'Ax - x'b$, the $A$-norm of the ...
- 205
6
votes
2
answers
539
views
Optimal polynomial approximation of rational function $\frac{1}{1-x}$
I've been working on the following polynomial approximation problem. I want to find the optimal Chebyshev approximation of the rational function $\frac{1}{1-x}$ on the real interval $x\in[-\rho, \rho]$...
- 63
0
votes
0
answers
63
views
Direct (first-order ?) algorithm to minimize $u(x) := \|x-a\|_C + r\|x\|_p$
Fix $a \in \mathbb R^n$, $r \ge 0$, $p \in \{1,2\}$, and a positive-definite matrix $C$ of order $n$. Define $u:\mathbb R^n \to \mathbb R$ by $u(x) := \|x-a\|_C + r\|x\|_p$, where $\|z\|_C := \sqrt{z^\...
- 6,853
0
votes
0
answers
580
views
Convex maximization over the boundary of a convex set
Let $f: \mathbb{R}^n \rightarrow \mathbb{R}$ be the objective function that is strictly convex. We would to like maximize $f$ over a convex compact set $S \subseteq \mathbb{R}^n$. Assume that $f$ has ...
- 261
2
votes
1
answer
101
views
Maximizing a skew-symmetric 4D cross product
How do I find two orthonormal 4D vectors, $(x_0,x_1,x_2,x_3)$ and $(y_0,y_1,y_2,y_3)$, which maximize this function:
$-19x_1y_0 - 33x_2y_0 + 11x_3y_0 + 19x_0y_1 - 21x_2y_1 - 33x_3y_1 + 33x_0y_2 + ...
- 1,547
3
votes
1
answer
158
views
Numerical scheme for convex optimization
Given $(e_n)_{-N\le n\le N}\in\mathbb R^{2N+1}$ and $-1<x<1$, solve
\begin{eqnarray}
&&\max_{(q_n)_{-N\le n\le N}\in\mathbb R^{2N+1}_+}~ \sum_{n=-N}^N (e_n-\log(q_n))q_n \\
\mbox{s.t.} &...
user128095
0
votes
1
answer
261
views
Non-asymptotic convergence rates for gradient descent
I'd like to know how the number of steps needed for gradient descent depend on properties of the Hessian in non-asymptotic regime.
More specifically, number of gradient descent steps needed to obtain ...
- 2,442
2
votes
0
answers
618
views
block diagonal approximation of (SPD) matrix
I am interested in approximating a symmetric matrix in a block diagonal form, i.e. compute just some entries of the matrix located in blocks around the diagonal. Are there any theoretical guarantees ...
- 335
2
votes
0
answers
40
views
Numerical algorithms for geodesically convex optimization
I want to solve a minimization problem of the form
$\inf_{x \in M} f(x)$
where $M$ is a Hadamard manifold and $f$ is geodesically convex (but not differentiable). Since I know that in general a ...
user85365
0
votes
0
answers
127
views
ADMM for solving linear systems
I would like to use ADMM for solving $Mx=b$, where $M\in \mathbb{R}^{R\times R}$ is symmetric and positive definite. I know that a lot of methods will do for me in this case, but I'm specially ...
- 133
4
votes
0
answers
509
views
analytic approximations of the min and max operators
Question:
What is the state of the art on analytic approximations of $\min$ and $\max$? My hunch is that numerical analysts probably have a better solution than the one I propose here.
For any $\...
- 3,871
0
votes
0
answers
68
views
Numerically solve a specific saddle-point problem
Let $(\Omega,\mathcal E,\mu)$ be a probability space, $k\in\mathbb N$, $$W:=\left\{w:E\to[0,\infty)^k:\sum_{i=1}^kw_i=1\;\mu\text{-almost surely}\right\},$$ $G$ be a finite nonempty set and $a^{(g)}:E\...
- 167
1
vote
0
answers
79
views
Minimization of a smooth integral functional over a closed convex set
Let $(E,\mathcal E,\mu)$ be a probability space, $I$ be a finite nonempty set, $\gamma:(E\times I)^2\to[0,\infty)$ be measurable, $$F_1(g,w):=\sum_{i\in I}\int\mu({\rm d}x)w_i(x)g(x)\sum_{j\in I}\int\...
- 167
2
votes
2
answers
323
views
Reference request on computational schemes for $\inf_{x\in\Omega^n}\sup_{y\in\mathbb R^n}F(x,y)$
Let $\Omega\subset \mathbb R^d$ be compact, $\rho$ be a density function on $\Omega$ and $p_1,\ldots, p_n\in (0,1)$ be weights satisfying $\int_{\Omega}\rho(z)dz=1=\sum_{k=1}^n p_k$. We consider the ...
user128095
1
vote
1
answer
1k
views
On solution methods for min-min optimization problems
Closely related (although not equivalent) to minimax optimization problems is the following:
$$\min_{x \in \Omega} \min_{i=1,...,q} f_i (x).$$ Here, $\Omega \subset \Bbb R^n$ and $f_i: \Bbb R^n \to \...
- 185
5
votes
3
answers
512
views
Optimisation under constraint of Wasserstein distance
Let $\mathcal P_n = \{P \in \mathbb R^n_{\geq 0}: P^T \mathbb I = 1 \}$, where $\mathbb I = (1,...,1)^T \in \mathbb R^n$ and $f: \mathcal P_n \to \mathbb R$ a convex and differentiable function (or ...
- 111
1
vote
0
answers
102
views
When are quadratic integer programs "easy to solve"?
Let $N_i=\{0,1,\dots,\bar{n}_i\}$ and define $N=N_1\times \dots \times N_I$. I want to maximize $f$ on $N$. $f$ has the following form
$$
f(n) = \sum_i A_i n_i -\sum_i \sum_{j\neq i} B_{ij} (n_i-n_j)^...
- 379
1
vote
0
answers
59
views
Lower semicontinuity of proximal gradient descent sequence
I'm using an alternate optimization scheme to minimize a function $F(a,b,c) = G(a,b,c) + H(c)$ where $G$ is continuous and differentiable, $H$ is not differentiable but lower semi-continuous and ...
- 11
2
votes
0
answers
88
views
Gradient Descent with derivative constraints
tl;dr: I need bound results for the derivative of some big honking function.
tfa: I am trying to solve an optimization problem:
Find a parameter vector $\theta$ so that $\sum_x \log f(\theta, x) \...
- 245
4
votes
1
answer
402
views
Linear convergence rate of proximal point algorithm
For $T : R^n \to P({R^n})$ maximally monotone, the proximal point algorithm (step size $c>0$)
$$
x^{k+1} = (I + c T)^{-1} x^k,
$$
converges linearly with rate $\kappa = \frac{1}{1 + c \sigma}$ if $...
- 303
4
votes
3
answers
283
views
Iterative matrix inversion with $L^\infty$ norm
The usual conjugate gradient type algorithms for iteratively finding the inverse of a matrix applied to a vector, $x = A^{-1} y$, works by minimizing $\|Ax - y\|^2$ where $\| \cdot \|$ is the $L^2$-...
- 1,150
4
votes
3
answers
848
views
algorithm for finding the minimizer of a almost convex function
Let $f(t)$ be a function from $(0,1)$ to $\mathbb R$. If $f$ is strictly convex, then finding the minimizer is an easy task. For example, newton's method would be able to do the job.
However, if my ...
- 679
1
vote
0
answers
227
views
Find optimal value for a regularization parameter in generalized eigenvalue problem
Consider the generalized eigenvalue problem :
$ \Sigma_{XY} \Sigma_{YX} {W} = \lambda \Sigma_{XX} {W} $
where $\Sigma_{XX} $ and $\Sigma_{XY}$ are sample covariance matrices are of the matrices $X$...
- 45
3
votes
2
answers
266
views
Fixed point iteration on symmetric biconvex function
Suppose $X\subseteq\mathbb{R}^n$ is a convex set and that a function $g(x,y):X\times X\rightarrow\mathbb{R}_+$ is smooth, "strictly biconvex" (strictly convex in $x$ and $y$ independently but not ...
- 705
2
votes
1
answer
2k
views
Projection onto $\ell^{2,1}$ ball
Does anyone have an idea how to project onto the $\ell^{2,1}$ ball efficiently, i.e. how to solve
$$ u = \arg \min_u \|u-f \|^2 \text{such that } \left(\sum_i \big(\sum_j |u_{i,j}|\big)^2 \right)^{...
- 41
7
votes
0
answers
209
views
Numerical linear algebra: how to compute $B^TC^{−1}B$ efficiently
Hi,
my question is similar to this one. I have to compute $B^TC^{−1}B$, where $C$ is a strictly positive definite $n\times n$ matrix and $B$ is $n\times m$.
The matrix $C$ is huge ($n$ up to a ...
- 619