5
$\begingroup$

I am interested in a measure for the quality of fit to a curve which would distinguish the two cases shown in the following image (without addressing the fact that incidentally the right one has more samples, but rather addressing the way it 'determines' the curve in a better way): For the sake of it let's say I want to fit a parabola to this data. Is there some well known measure for this?

enter image description here

$\endgroup$
3
  • $\begingroup$ I guess the sums of the squares of the distances between your curve and your points is not a good measure? That's the standard measure used in least-squares fitting... $\endgroup$ Dec 25, 2009 at 23:40
  • $\begingroup$ Mariano, I guess that "sums of squares of distances" would judge the second curve to be a worse fit, simply because there are more sample points. You could ask for the mean squared distance, but that would probably give about the same for both curves. $\endgroup$ Dec 26, 2009 at 0:17
  • $\begingroup$ Incidentally, it's a naive-sounding question, but seems to me to be a good one. Statisticians and numerical analysts must have thought hard about it, and I'd be interested to know whether there's a single measure of quality of fit that is agreed to reign supreme, or whether there are many competitors and the choice of which to use is seen as subjective. $\endgroup$ Dec 26, 2009 at 0:22

3 Answers 3

5
$\begingroup$

If you use the least square fit, the second case may have a better conditioned matrix but this measure may be hard to compute in practice. Still, it is going to be something in this venue because the story is not about the second curve being a better approximation than the first but about its being "more unique", so to say, and that is exactly what the condition number measures for solutions of linear systems.

Of course, since we are talking about approximate solutions, not exact ones, we may want to modify the notion of the condition number a bit. One possible quantity that seems relevant to the "approximate uniqueness" is the following: the least square problem is just about minimization of a quadratic form $Q(x)$ and if $y$ is the solution, then $Q(x)=Q(y)+(A(x-y),(x-y))$. Now, we want to see what is the penalty for going away from the optimal vector. So, both $\frac{\mbox{Tr\,}A}{Q(y)}$ and $\frac{\mu(A)}{Q(y)}$ where $\mu(A)$ is the least eigenvalue of $A$ seem to make sense as measures of such penalty. The higher is this number, the more unique is the approximation. The reason for the denominator is that I wanted to measure the sizes of deviations that change the minimum by certain percentage. You may want to do the absolute error instead, or something else. It may be a good idea to figure out what invariance properties you want from your measure first. For instance, should it be invariant with respect to stretchings or you think that two close points determine a line less precisely than two distant points?

$\endgroup$
2
$\begingroup$

If you use linear regression to calculate such parabolas, then the Pearson correlation coefficient is a good measure of the quality of the fit.

$\endgroup$
1
$\begingroup$

If I understand correctly, you would like to show that the right-hand dataset better defines the curve? There are many ways you could go about this, here's two:

A) You show that one dataset leaves less uncertainty about the parameters of the curve than the other. In a classical setting you'd fit the parameters using some kind of regression, and compare the error bars on the fitted parameters. In a Bayesian setting you could compare the two posterior distributions, maybe showing that one has lower posterior entropy than the other.

B) You show that one dataset leads to better predictive ability. Use cross-validation to compute an estimate of the generalisation error in each dataset, and compare that.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.