Timeline for Levenberg-Marquadt near the minima for non-zero-residual problems
Current License: CC BY-SA 3.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 5, 2012 at 18:01 | comment | added | Brian Borchers | @Alex; The $J^{T}J$ approximation to the hessian of $c(x)$ works well when $f_{i}(x)-y_{i}$ is small, but if the residuals are large then the approximation can degrade. This is because the second order terms that go into the Hessian are dropped when we approximate it with $J^{T}J$, and those terms have $f_{i}(x)-y_{i}$ factors. | |
| Sep 5, 2012 at 16:32 | vote | accept | Alex Flint | ||
| Sep 5, 2012 at 14:07 | comment | added | Alex Flint | @ThomasKlimpel Thanks, this is helpful. Is there a text you can recommend on this topic? I've consulted a couple of convex optimization books but they (understandably) do not go into this level of detail about this specific setting. Also, can you elaborate on "LM is not very efficient for this type of problem" - my problem is certainly overdetermined and I do not expect a perfect fit. I also expect there to be (infrequent) outliers, which I expect to have large residuals at the optimum (I am using a robust cost function for this reason). | |
| Sep 4, 2012 at 8:37 | comment | added | Thomas Klimpel | @BrianBorchers Considering that AlexFlint writes that the zero is only occurring after a few steps, your explanation might be closer to what is actually happening in his case than my suggestion. However, if this should be the case, then there is something strange going on. I believe this can happen if the model fitting problem degenerates into an optimization problem, i.e. if the model can't really fit the measurement data, and has to try to maximize (or minimize) the model component which can't be fitted. But note that LM is not very efficient for this type of problem. | |
| Sep 4, 2012 at 0:40 | comment | added | Brian Borchers | "It just means that the problem doesn't depend on the corresponding variable" isn't a correct statement. Consider the example of minimizing $(x_{1}^{2}-0)^{2}+(x_{2}^{2}-0)^{2}$. Clearly, $x=0$ is the unique optimal solution, but $J^{T}J$ is 0 at $x=0$. | |
| Sep 3, 2012 at 22:32 | history | answered | Thomas Klimpel | CC BY-SA 3.0 |