Timeline for Is the pseudoinverse the same as least squares with regularization?
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 26, 2021 at 7:14 | history | edited | Federico Poloni | CC BY-SA 4.0 |
made additional conditions clearer to answer the comments
|
| Feb 26, 2021 at 7:05 | comment | added | Federico Poloni | @YaakovBaruch I agree; but this can be fixed by agreeing that the second problem is also formulated a double minimization "$\min \|x\|$ such that $x$ minimizes...". Indeed the second problem is not stated formally as well as the first in the original question; just the objective function appears. (That formulation, however, may be considered a bit artificial, because the minimizer of $\|Ax-b\|^2+\lambda \|x\|^2$ is unique unless $\lambda=0$.) | |
| Feb 25, 2021 at 23:57 | comment | added | Yaakov Baruch | Beautiful reduction. But I think @HermanJaramillo has a point: you are proving that minimizing the LHS is equivalent to minimizing the RHS. But when $\lambda=0$ the original problem is not simply to minimize the LHS (which may have infinite solutions) but to pick the unique shortest minimizer. | |
| Feb 25, 2021 at 22:29 | comment | added | Federico Poloni | $\begin{bmatrix}A \\ \sqrt{\lambda} I\end{bmatrix}$ always has trivial nullspace, because of that identity block. | |
| Feb 25, 2021 at 22:12 | comment | added | Herman Jaramillo | When $\lambda=0$ the second does not reduce to the first, if the null space of $A$ is not the trivial. That is, if the columns of $A$ are linearly dependent, the least square problem does not have unique solution and we need to solve, in addition the $\min \| x \|$ problem. I understand your second equation but this does not show that $x \perp \mathcal{N}(A)$ as it should be if we are finding the pseudo-solution. | |
| Feb 25, 2021 at 14:46 | history | answered | Federico Poloni | CC BY-SA 4.0 |