MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm optimizing a function of the form

$$ \sum \frac{ \|\mathbf{f_i}(x)\|^2 }{ g_i(x)^2 + h_i(x)^2 } $$

where $x$ is a real vector, $\mathbf{f}(x)$ is a real vector, and $g(x)$ is a scalar. My first thought is to use something from the Gauss-Newton family, in which I would write

$$ \mathbf{c_i}(x_i) = \frac{ \mathbf{f_i}(x) }{ \sqrt{g_i(x)^2 + h_i(x)^2} } $$

$$ C(x) = \sum \|\mathbf{c_i}(x)\|^2 $$

and from there I would take gradient steps, which would be like approximating each $c_i$ as locally linear. But while this would be valid, I'm wondering if I could do better by exploiting the known structure of the cost function. In particular, I'm wondering if I can use the gradients of $\mathbf{f}$, $g$, and $h$ directly. Particularly if the curvature of $f$, $g$, and $h$ is small compared to the curvature of the square root function, it would surely be advantageous to take account of the gradients of $f$, $g$, and $h$ directly. Can anyone point me in the right direction?

share|cite|improve this question
I could be wrong, but I doubt very much you can do better than using the overall gradient (which already is a weighted sum of the gradients of $f$, $g$, and $g$). – Deane Yang Aug 31 '12 at 0:44
My thought was that the Gauss-Newton algorithm approximates the Hessian by assuming that the function to be optimized looks like a sum of squared residuals. Although I can write my function as a sum of squared residuals, perhaps there is a better way to approximate the Hessian taking notice of the specific structure of my cost function. – Alex Flint Sep 2 '12 at 16:47

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.