# Uniqueness of solution of a nonconvex optimization problem

What conditions need to be hold for a nonconvex optimization problem to have a unique solution?

Specifically, I have the following minimization problem that I'd like to know whether it has a unique solution or not:

$\min_{\bf A, B} \| {\bf Y - XAB} \|_2^2 + \lambda_1 \|{\bf A}\|_1 + \lambda_2 \|{\bf B}\|_1$

where ${\bf Y}_{n \times q}$ and ${\bf X}_{n \times p}$ are fixed, and ${\bf A}_{p \times r}$ and ${\bf B}_{r,q}$ are the variables ($r \leq \min(p,q)$).

Also, I know that this minimzation problem is convex if either ${\bf A}$ or ${\bf B}$ are fixed (and so it has a unique solution in that case).

Is there a certain relaxation that transforms this into a convex problem?

• It would help if you would specify the meaning of the notation $\|\cdot\|_1$, since there are at least three matrix norms which often use that same notation. But, at any rate, at least for the three I have in mind, ... – cardinal Nov 20 '14 at 1:55
• Hint: Think about the set of $r \times r$ permutation matrices. – cardinal Nov 20 '14 at 1:56