1
$\begingroup$

I have the following optimization problem (like the LASSO problem but with maximization instead of minimization):

$\mathbf{maximize}_{\boldsymbol{\alpha}} \|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2$ s.t. $\|\boldsymbol{\alpha}\|_0 \leq k$

with $\alpha$ a sparse vector with no more than $k$ non-zero elements.

We can notice I have a maximization problem instead of a minimization one. To solve this problem, I have first converted into a minimization problem:

$\mathbf{minimize}_{\boldsymbol{\alpha}} -\|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2$ s.t. $\|\boldsymbol{\alpha}\|_0 \leq k$

What I want first to know is if my above minimization problem is correct or not?

I also would like to know if it is possible to solve directly the maximization problem without converting it into a minimization one? If it is possible, so please can you provide me the detailed calculation?

Then, I can proceed as follows:

$\mathbf{minimize}_{\boldsymbol{\alpha}} -\|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2 + \lambda \|\boldsymbol{\alpha}\|_1$.

This problem can be simply solved via alternating direction method of multipliers. For example by introducing an auxiliary vector $\boldsymbol{\beta}$ such that $\boldsymbol{\alpha} = \boldsymbol{\beta}$:

$\mathbf{minimize}_{\boldsymbol{\alpha}, \boldsymbol{\beta}} -\|\mathbf{x} -\mathbf{A}\boldsymbol{\alpha}\|^2 + \lambda \|\boldsymbol{\beta}\|_1$ s.t. $\boldsymbol{\alpha} = \boldsymbol{\beta}$.

Is that correct? Do I have some special trick to solve the above maximization problem?

$\endgroup$
2
  • $\begingroup$ Notice that the correct LaTeX code for a double bar is not || but \|. $\endgroup$
    – Alex M.
    Commented Sep 24, 2018 at 15:58
  • $\begingroup$ @AlexM. Thank you for your time in editing my question. $\endgroup$
    – Christina
    Commented Sep 24, 2018 at 18:48

1 Answer 1

2
$\begingroup$

That problem does not have a maximum. Unless $A$ or $k$ are zero, you can take $\alpha = Me_j$, where $e_j$ is a vector of the canonical basis, and then the objective function diverges when $M \to \infty$.

$\endgroup$
3
  • $\begingroup$ Thank you for the answer. Can you please write in your answer how my new optimization problem will become in more details by applying your proposition? $\endgroup$
    – Christina
    Commented Sep 24, 2018 at 18:46
  • 2
    $\begingroup$ Your optimization problem doesn't become anything. It is ill-posed. You need to go back to the modelling phase, double-check that all your signs are correct and make sense, and think about adding another penalty term if that's really what you want to solve. $\endgroup$ Commented Sep 24, 2018 at 18:53
  • $\begingroup$ Yes I see. I will try to edit my whole question very soon. I will keep you informed. Thank you a lot dear Federico! $\endgroup$
    – Christina
    Commented Sep 24, 2018 at 19:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .