I'm trying to fit a quadratic $a_0 + a_1x + a_2x^2$ by Polynomial Regression:

$$ \begin{pmatrix} n & \Sigma x_i & \Sigma x_i\\ \Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3\\ \Sigma x_i^2 & \Sigma x_i^3 & \Sigma x_i^4\\ \end{pmatrix} \begin{pmatrix} a_0\\ a_1\\ a_2\\ \end{pmatrix}= \begin{pmatrix} \Sigma y_i\\ \Sigma x_iy_i\\ \Sigma x_i^2y_i\\ \end{pmatrix} $$ by Gaussian Elimination on the augmented matrix: $$ \left[ \begin{array}{ccc|c} n & \Sigma x_i & \Sigma x_i & \Sigma y_i\\ \Sigma x_i & \Sigma x_i^2 & \Sigma x_i^3 & \Sigma x_iy_i\\ \Sigma x_i^2 & \Sigma x_i^3 & \Sigma x_i^4 & \Sigma x_i^2y_i\\ \end{array} \right] $$

I'm not a matrix whizz, so how do I solve this using the same method when I know I want the y-intercept of the curve, $a0$, to be 0?

  • $\begingroup$ Regarding some of your comments below, see mathoverflow.net/help/on-topic -- the StackExchange tag line is not entirely accurate, and MathOverflow actually predates MathStackExchange which is the more appropriate site for questions like this $\endgroup$
    – Yemon Choi
    Jul 25 '14 at 18:40
  • $\begingroup$ @YemonChoi Why am I not able to delete my answer? Is it because the question is closed, or because the answer is accepted? $\endgroup$
    – M. Vinay
    Jul 26 '14 at 4:00
  • $\begingroup$ @M.Vinay I'm not sure - it could be either or both of those reasons $\endgroup$
    – Yemon Choi
    Jul 26 '14 at 12:08

If you want $a_0$ to be $0$, then the curve you are trying to fit has equation $y = a_1 x + a_2 x^2$. If you derive the normal equations starting from this equation, you get

$a_1 \sum x_i^2 + a_2 \sum x_i^3 = \sum x_i y_i\\ a_1 \sum x_i^3 + a_2 \sum x_i^3 = \sum x_i^2 y_i$


$\left[\begin{array}{cc|c} \sum x_i^2 & \sum x_i^3 & \sum x_i y_i\\ \sum x_i^3 & \sum x_i^4 & \sum x_i^2 y_i \end{array}\right]$

Derivation of Normal Equations
$y = a_1 x + a_2 x^2$

The error $\epsilon_i$ in the $i$th term is the difference between the predicted value $y(x_i)$ and the observed value $y_i$:
$\epsilon_i = y(x_i) - y_i = a_1 x_i + a_2 x_i^2 - y_i$

The total error is the sum of squares of these errors:
$E = \displaystyle\sum_i \epsilon_i^2 = \displaystyle\sum_i(a_1x_i + a_2x_i^2 - y_i)^2$

To find the values of the parameters $a_1$ and $a_2$ that minimize the total error, let $\dfrac{\partial E}{\partial a_1} = \dfrac{\partial E}{\partial a_2} = 0$:

$ \dfrac{\partial E}{\partial a_1} = \sum 2x_i(a_1 x_i + a_2 x_i^2 - y_i) = 0 \Rightarrow a_1 \sum x_i^2 + a_2 \sum x_i^3 = \sum x_i y_i\\ \dfrac{\partial E}{\partial a_2} = \sum 2x_i^2(a_1 x_i + a_2 x_i^2 - y_i) = 0 \Rightarrow a_1 \sum x_i^3 + a_2 \sum x_i^4 = \sum x_i^2 y_i\\ $

  • $\begingroup$ This website is for questions about math research. Please don't answer questions that don't belong here. $\endgroup$ Jul 25 '14 at 11:37
  • 1
    $\begingroup$ @EdKing He's right, actually: "MathOverflow's primary goal is for users to ask and answer research level math questions". I knew this, but still answered as I didn't see any comments or downvotes on the question. You should have asked this on Mathematics StackExchange, though. $\endgroup$
    – M. Vinay
    Jul 25 '14 at 13:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.