Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Ok, so I am running a classic linear regression where betahat = (X'X)^-1X'y

Due to performance issues, I would like to estimate betahat with an additional data point (x1,x2,x3,x4,...,y) without recalculating based on the whole history.

Could I do some sort of multiplication based on Xmu or std deviation of the variables? And then what is the probability that betahat is predicting the true beta from this point? This would be used so that I can judge when I need to do a full recalculation.

Or am I thinking about this entirely wrong?

Thank you for any help or suggestions, Jeremy

share|improve this question
add comment

closed as off-topic by Ricardo Andrade, Chris Godsil, Stefan Kohl, j.c., Daniel Moskovich Dec 13 '13 at 0:15

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question does not appear to be about research level mathematics within the scope defined in the help center." – Ricardo Andrade, Chris Godsil, Stefan Kohl, j.c., Daniel Moskovich
If this question can be reworded to fit the rules in the help center, please edit the question.

2 Answers

There's a very large literature on updating solutions to least squares problems as new data are added. The naive formula $\hat{\beta}=(X^{T}X)^{-1}X^{T}y$ can be problematic in practice because of numerical issues with ill-conditioning. The QR factorization is typically a much better choice in practice. There are lots of papers on updating the QR factorization as data are added. See for example:

http://eprints.ma.man.ac.uk/1192/01/covered/MIMS_ep2008_111.pdf

share|improve this answer
add comment

You might look into the Kalman filter http://www.lce.hut.fi/~ssarkka/course_k2012/handout2.pdf to do the update.

share|improve this answer
add comment

Not the answer you're looking for? Browse other questions tagged or ask your own question.