Skip to main content
3 of 3
added 596 characters in body
Mark L. Stone
  • 1.5k
  • 1
  • 10
  • 17

Solve the unconstrained least squares problem in "one-shot", for example by QR or SVD (if not too big), if you consider that to be "one-shot". Then if the optimal $x$ to the unconstrained least squares problem satisfies $A x \le b$, it is optimal for the original constrained problem; if not, you have not solved the original problem in one-shot.

Of course it is possible to solve the original constrained problem in one-shot if the one-shot consists of querying an oracle which can solve such problems. The only difficulty from a practical engineering perspective is that you may not have such an oracle available - I don't happen to.

The question title mentions convex/non-convex. But the only the problem you displayed is convex, and is a linearly constrained linear least squares problem, which can also be viewed as being a convex Quadratic Programming (QP) problem. Or it can be transformed into an equivalent Second Order Cone Problem (SOCP) via epigraph reformulation, which may be numerically advantageous from a robustness standpoint.

$min_{t.x} t$

s.t. $\|y-x\|_2 \le t, A x \le b$

Such linear inequality constrained linear least squares problems, linear inequality constrained QPs, and SOCPs are not solved in "one-shot", as you seem to define shot.

If $y$ were a nonlinear function of $x$, then you would have a linearly constrained nonlinear least squares problem which might be non-convex.

There are specialized, efficient, and robust numerical optimization solvers available for linearly constrained linear least squares, (linear inequality constrained) QPs, and for SOCPs.

The way you have written the problem, the condition number of the Hessian of the objective function is 1, so there is no harm in squaring the condition number of the problem via explicit QP formulation. More generally, there is harm in squaring the condition number via explicit QP formulation, though. In such case, a change of variables via Cholesky faotrization can be introduced such that the objective Hessian of the transformed problem is the Identity matrix (as is apparently the case in your problem to begin with), and a matrix equality constraint added defining the transformation.

Mark L. Stone
  • 1.5k
  • 1
  • 10
  • 17