All Questions
Tagged with linear-algebra convex-optimization
121 questions
20
votes
4
answers
5k
views
Is the pseudoinverse the same as least squares with regularization?
Given a linear system $Ax=b$, the pseudoinverse of $A$ is found as the matrix $A^+$ such that $x=A^+ b$ where $x$ solves the least squares problem $\min \| Ax - b \|^2 $ and $x \perp \mathcal{N}(A)$. ...
- 501
8
votes
3
answers
526
views
Lower bound for spectral radius on $\operatorname{GL}(n,\mathbb{Z})$
Consider the group of matrices $G =\operatorname{GL}(n,\mathbb{Z})$ with integer entries and determinant $\pm 1$. For each matrix $D \in G$, the product of the eigenvalues of $D$ is equal to $\det D =\...
- 345
8
votes
2
answers
809
views
Bounding the spectral gap of a simple symmetric matrix
I have a seemingly innocent linear algebra problem that I cannot solve, and which I hope that you would kindly offer some insight into. Here is the description: Let $\mathbf{a} = (a_1, a_2, \dots, a_d)...
- 333
8
votes
3
answers
2k
views
Optimization problem with determinant as objective
Let $A$ be a given symmetric positive definite $N\times N$ matrix. I need to find a symmetric positive semi-definite matrix $S$ which is the solution to the following optimization problem
\begin{align}...
- 1,421
8
votes
1
answer
2k
views
Finding Toeplitz matrix nearest to a given matrix
For an arbitrary $N\times N$ Hermitian matrix $A$, I want to derive a Toeplitz matrix from $A$ such that the eigenvectors of both matrices have minimal change.
Specifically I want find the Toeplitz ...
- 495
7
votes
1
answer
373
views
Is the solution of this optimization problem always positive semidefinite?
We are given a set of unit vectors $U \subset \mathbb{C}^n$ which spans the space $\mathbb{C}^n$. Given another unit vector $x$, consider then the following optimization problem:
$$ \sup_H \left\{ x^*...
- 73
7
votes
2
answers
251
views
What methods do we have to understand the spectrum of matrices with restricted entries?
Consider questions of the form (or the "most probable value of" version of these questions rather than the "largest possible"),
What is the largest possible spectral radius of a $...
- 1,893
7
votes
0
answers
1k
views
Closed-form solution of a linear programming question
Among all the probability matrices
\begin{equation*}
P =
\left(\begin{array}{cccc}
p_{00} & p_{01} & \ldots & p_{0,J-1} \\
p_{10} & p_{11} & \ldots & p_{1,J-1} \\
\vdots & \...
7
votes
0
answers
217
views
Characterizing matrices with rank constraint
Given matrix $M\in\Bbb\{0,1\}^{n\times n}$, is there a nice method to characterize $$\mathscr{D}[M]=\{Q\in\Bbb\{0,1\}^{n\times n}:\mathsf{rank}(M-Q)= \mathsf{rank}(Q),\quad M-Q\in\Bbb\{0,1\}^{n\times ...
- 13.9k
7
votes
0
answers
209
views
Numerical linear algebra: how to compute $B^TC^{−1}B$ efficiently
Hi,
my question is similar to this one. I have to compute $B^TC^{−1}B$, where $C$ is a strictly positive definite $n\times n$ matrix and $B$ is $n\times m$.
The matrix $C$ is huge ($n$ up to a ...
- 619
6
votes
1
answer
4k
views
Complexity for solving linear equations?
What is the best known complexity for finding a vector $x \in \mathbb{R}^n$ to minimize $||Ax - b||^2$ and/or to solve (when possible) the system of linear equations $Ax=b$?
I am interested in ...
- 544
6
votes
2
answers
5k
views
Minimize Frobenius norm
My question is the following:
Suppose $M$ is an $n \times n$ symmetric real matrix. I want to find an $n \times n$ symmetric real matrix X such that $|| X -M||_F$ is minimized with the constraint ...
- 263
6
votes
2
answers
875
views
Is there any efficient solution of the matrix equation AXB + (AXB)' + cX = D
I am trying to find the symmetric solution $X\in \mathbb{R}^{p\times p}$ of following matrix equation:
$AXB + (AXB)^T + cX = D$
where $A,B\in \mathbb{R}^{p\times p}$ are symmetric positive ...
- 89
6
votes
0
answers
136
views
Minimizing $\det(D)$ for all diagonal matrices $D$ that satisfy $D+B \succeq 0$
Let $A$ be an $n \times n$ real matrix and let $B$ be the block bipartite matrix
$$B = \begin{bmatrix} 0&A \\
A^{T}&0 \end{bmatrix}$$
I came across the following optimization problem, which ...
- 401
5
votes
3
answers
496
views
Eigenvectors that are tensor products?
Consider a fixed $N\times N$ positive definite symmetric matrix $A$.
Assume $N=d^r$ for some $d,r\geq 1$.
I wonder if one can find a closed formula for the maximizer/maximum of the function $$f(x):=\...
- 2,135
5
votes
1
answer
955
views
Maximizing Frobenius Norm of Commutator (an opposite Procrustes problem)
I was wondering if anybody has any suggestions on the following problem:
Let $S$ be an $n\times n$ positive definite symmetric matrix. I wish to find an $n\times n$ orthogonal matrix $R$ which ...
- 617
5
votes
0
answers
330
views
Best Approximation in Operator/non-Frobenius Norm
Since the Frobenius norm on matrices is generated by an inner product, solving the optimization/approximation problem of approximating an operator $X$ with a scalar multiple of another operator $Y$
$$\...
- 153
4
votes
1
answer
175
views
Maximizing trace subject to two equality constraints
I am looking at the following optimization problem
$$\begin{align}
\underset{{\bf X}}{\text{maximize}} \qquad&\mathrm{tr}({\bf AX})\\
\text{subject to} \qquad& \mathrm{tr}({\bf X}) = 1,\\
&...
- 43
4
votes
2
answers
734
views
Fast projection onto a subspace
Given an $n$-dimensional vector $\mathbf{c}\in [0,1]^n$, let $\Delta_{\mathbf{c}}$ be the set of points $\{\mathbf{x}\in [0,1]^n: \langle \mathbf{c},\mathbf{x} \rangle \le 1\}$, where $\langle \mathbf{...
- 1,677
4
votes
2
answers
604
views
A certain type of quadratic constrained quadratic program (QCQP)
Let $P_1$, $P_2$ be two Hermitian matrices. Can anyone comment on the following QCQP?
$$\begin{array}{ll} \text{minimize} & z^{H} z\\ \text{subject to} & z^{H} P_1 z +1 \leq 0\\ & z^{H} ...
- 1,421
4
votes
1
answer
456
views
Least square solution to $AXB+CXD=E$
I am trying to find the least-squares solution $X$ of the following matrix equation
$$AXB+CXD=E$$
Of course, I know that this equation can be written in the form
$$(B^T \otimes A+D^T \otimes C) \...
- 191
4
votes
3
answers
283
views
Iterative matrix inversion with $L^\infty$ norm
The usual conjugate gradient type algorithms for iteratively finding the inverse of a matrix applied to a vector, $x = A^{-1} y$, works by minimizing $\|Ax - y\|^2$ where $\| \cdot \|$ is the $L^2$-...
- 1,150
4
votes
1
answer
119
views
Is the Loewner maximum uniquely defined?
Given 2 (symmetric) PSD matrices $A,B$, is the following set $S_{A,B}$ non-empty?
$$ S_{A,B} = \{ C: C\succeq A, C\succeq B, \text{ and }\forall D, D\succeq A, D\succeq B \implies D\succeq C \} $$
If ...
- 41
4
votes
2
answers
359
views
A certain type of constrained Rayleigh-Ritz ratio
Let $\mathbf{A_1}$ and $\mathbf{A_2}$ be two hermitian matrices. Consider the problem
\begin{align}
\max_{\mathbf{u}^H\mathbf{u}=1}~\mathbf{u}^H\mathbf{A}_1\mathbf{u} \\\
\mathbf{u}^H\mathbf{A}_2\...
- 1,421
4
votes
1
answer
2k
views
lipschitz constant of a multivariate function
I have a function $f:\mathbb{R}^{50} \rightarrow \mathbb{R}$ and I need to compute the Lipschitz constant of $f$ to solve an optimization problem using a specific algorithm. Does any one have ...
- 41
4
votes
0
answers
307
views
Derivative of rank $r$ approximation of matrix
Let $Y \in \mathbb R^{n \times c}$ and $r$ be an integer with $1 \le r \le \operatorname{rank}(Y)$. Consider the problem
$$\text{minimize} \|Y-X\|_{\text{Fro}}^2\text{ over }X \in \mathbb R^{n \times ...
- 6,853
4
votes
0
answers
205
views
Basin of Attraction
I have a function $F$ which is defined as follows:
$$
F(x) = \sum_{i=1}^N f(z_i^T x)
$$
where ${z_i}$ are known $m \times 1$ vectors, $x$ is an $m \times 1$ vector, and for $t\in \mathbb{R}$, $f(t) = \...
- 291
4
votes
0
answers
126
views
An inequality from the "Interlacing-1" paper
This question is in reference to this paper, http://annals.math.princeton.edu/wp-content/uploads/annals-v182-n1-p07-p.pdf (or its arxiv version, http://arxiv.org/abs/1304.4132)
For the argument to ...
3
votes
2
answers
380
views
How to find a solution of a large system of linear diophantine inequalities?
I need to find a solution (all solutions, or at least upper and lower bounds) in positive integer numbers to the system $Ax \ge f$, where $A$ is an integer matrix.
With SageMath, I solved it with the ...
3
votes
1
answer
275
views
Uniqueness of l1 minimization
Let $A \in \mathbb{R}^{m \times n}$.
Is it true that $$\min \limits_{Q \in \mathbb{R}^{n \times m}}|I - QA|_{\infty} < \frac{1}{2}$$ is criteria for the uniqueness of the 1-sparse solution to
$\...
- 33
3
votes
1
answer
4k
views
Schur complement and negative definite matrices
Hello,
My question regards to the Schur complement lemma. Consider the matrix $M=\left( \begin{array}{cc}
A & B\\\
B^T & C \end{array}\right)
$.
According to the lemma $M\geq0$ iff $C>0$ ...
- 155
3
votes
1
answer
775
views
Is there some quantitative version of interlacing of eigenvalues of a matrix under rank-1 update?
Given a real symmetric matrix $A$ and a vector $v$ of the same dimension we know that the eigenvalues of $A + vv^T$ are left interlaced by the eigenvalues of $A$.
But do we have any quantitative ...
- 1,893
3
votes
1
answer
660
views
Condition number after preconditioning
Suppose $A$ and $P$ are symmetric, positive definite matrices and that we factor $P^{-1}=EE^\top.$ Is it true that the condition number of $PA$ is upper-bounded by the condition number of $E^{-1}AE^{-...
- 705
3
votes
1
answer
2k
views
Fast algorithm for maximizing smallest eigenvalue of linear combination of hermitian matrices
I have an engineering back ground. Due to work, I came across this problem
\begin{align}
&\max_{\lambda,y_i\in \mathbb{R}}~\lambda \\\
s.t.~&\left(\mathbf{A}_0+\sum_{i=1}^{K}y_i\mathbf{A}_i\...
- 1,421
3
votes
2
answers
215
views
Is there a closed-form solution for $\max_D \operatorname{Tr}(ADBD)$
Is there a closed-form solution for
$$\max_D \operatorname{Tr}(ADBD)$$
where $D$ is a $N\times N$ diagonal matrix with $m<N$ number of $1$'s and the rest are $0$'s, and $A$ and $B$ are real ...
- 433
3
votes
1
answer
704
views
Find the following transformation $G$
I asked this question 6 days ago on math.stackexchange.com (https://math.stackexchange.com/questions/656585/find-the-following-transformation-g). I didn't get any answers yet, so I'm posting here.
I'...
- 53
3
votes
0
answers
122
views
Convex optimization upper bound for a non-linear optimization
Is there any good convex optimization problem based upper-bound for the following non-linear optimization problem?
\begin{align}
\max_{x_1,\ldots,x_N}&\quad \sum_{n=1}^{N} \log(1+\frac{x_n}{1+\...
- 287
3
votes
0
answers
116
views
convex approximation for a non convex function
Consider the function
$f\left( {{x_1},...,{x_M},{y_1},...,{y_N}} \right) = \left( {\sum\limits_{j = 1}^M {{\alpha _j}{x_j}} } \right)\left( {{e^{ - \sum\limits_{i = 1}^N {{\beta _i}{y_i}} }}} \right)$...
- 275
3
votes
1
answer
154
views
Solving Matrix/Operator Equation $H P X + X P H + HQ = 0$
This problem arises when minimizing the operator equation $X P X^* + X Q + R$ over positive $X$ with respect to the positive cone on a Hilbert space $\mathcal{H}$.
The (reduced) task:
Given $P$ and $...
- 153
2
votes
2
answers
352
views
The set of matrices with same spectral radius
I am working on an optimization problem over the set of positive matrices (that is, matrices where all entries are positive numbers) that have the same spectral radius. My main problem is how to ...
- 51
2
votes
1
answer
423
views
Simple proof for convexity of a real valued matrix function
I am looking for a simple and short proof showing that $X \to \|X X^\top\|_F^2$ is a convex function where $\|\cdot\|_F$ is the Frobenius norm. I have one proof by showing that the derivative is ...
- 407
2
votes
1
answer
1k
views
$\arg\max$ in the dual norm of the nuclear norm
Given a matrix $X \in \mathbb{R}^{m \times n},$ then the spectral norm is defined by
$$\left \| X\right\| := \max\limits_{i \in \{1, \dots, \min\{m,n\}\} }\sigma_i (X)$$
whereas the nuclear norm is ...
2
votes
1
answer
159
views
Conic hull of a rectangle
I have a simple question that appeared in research: For a rectangle $S :=[a_1,b_1] \times[a_2,b_2] \times \dots \times [a_n,b_n] \subset \mathbb{R}^n$. Let $p_0 = (a_1,a_2,\dots,a_n)$, and define $p_i ...
- 275
2
votes
2
answers
99
views
Extremal Covariance Duality
Given real vectors $v$ and $r$ of the same size, what are the following?
$\inf\{v'R^{-1}v ~ \colon ~ R>0 \, , \, \text{diag}(R)= r\}$
$\sup\{v'Rv ~ \colon ~ R>0\, , \, \text{diag}(R)= r\}$
...
- 81
2
votes
2
answers
107
views
Solution to a matrix optimisation problem with a particular structure
Does a matrix of the form $A_{ij} = v_i + v_j$ for some arbitrary vector $v$ have a particular name?
I am attempting to find the closed form solution (if it exists, although it looks like it might) ...
- 31
2
votes
1
answer
226
views
Tighest Gap $\|x\|_1/\|x\|_2$ between $\ell^1$ and $\ell^2$ norms
I'm looking specifically at the optimization problem
$$
\begin{align*}
\text{maximize: }& n - \frac{\|\lambda\|_1^2}{\|\lambda\|_2^2}\\
\text{subj. to: }& \lambda \succeq \epsilon\mathbf{1}
\...
- 153
2
votes
2
answers
2k
views
Hessian of function of covariance matrices
Suppose we have a typical logdet function $\mathcal{L}$ with respect to a covariance matrix $\mathbf{A}$,
$$
\mathcal{L}(\mathbf{A}) = \log\vert \mathbf{I} + \mathbf{A}\mathbf{S} \vert - \mathbf{q}^T(\...
- 125
2
votes
2
answers
104
views
Inequality for matrix with rows summing to 1
Let $A$ be real matrix with $M > 1$ rows and $K > 2$ columns, and each entry $a_{m,k} \in (0,1)$, with each row summing to $1$. For all $m$
$$
\sum_{k=1}^{K} a_{m,k} = 1
$$
I want to find out if ...
- 23
2
votes
1
answer
189
views
Analytic expression for the Moreau envelope of $x \mapsto \|Ax\|$
Given an $m \times n$ matrix $A$ and a vector $c \in \mathbb R^n$, define $\eta(A,c) \ge 0$,
$$
\eta(A,c) := \sup_{u \in \mathbb R^n} u^\top c - \frac{1}{2}\|u\|^2 - \|Au\|.
$$
Note that $\eta(A,c) = \...
- 6,853
2
votes
1
answer
344
views
Joint convexity of trace of matrices
Let $\Gamma_{m\times m}$ be a diagonal matrix with positive diagonal entries and $\mathbf{A}_{m\times m}$ be an arbitrary matrix. Then, is the following trace function jointly convex on $\Gamma_{m\...
- 287