3
$\begingroup$

I wanted to learn prediction, forecasting etc. I also have time series data on millions of online videos. I would like to test out prediction algorithms etc on this data set, for eg. Linear Prediction, Kalman filter.

Are there any good resources out there to get me started on those?

Edit: Let me rephrase the question. If you were given such a time series data set with a million videos with some number of attributes to each, what steps/path path would you take to come up with a decent view prediction scheme? For example, factor analysis or PCA etc first.

$\endgroup$
4
  • $\begingroup$ I suspect you'll find this question is way off the applied end of the expertise of participants here. $\endgroup$ Nov 13, 2009 at 18:09
  • $\begingroup$ what do you mean by view prediction scheme exactly? $\endgroup$ Nov 14, 2009 at 4:27
  • $\begingroup$ and there are some applied math people here... well, at least I am (thought this isn't my area... I've been working on this but am definitely no expert) $\endgroup$ Nov 14, 2009 at 4:29
  • $\begingroup$ @michael-hoffman, by view prediction scheme i meant if you wanted to predict the number of views a random video will get over the next week given its attributes in the current week. Although this is difficult to do, but how would i go about minimizing the error in such a case. $\endgroup$
    – sidmitra
    Nov 14, 2009 at 18:38

4 Answers 4

2
$\begingroup$

The following might be of some help

linear regression

Bayesian inference

(possibly).

$\endgroup$
1
  • $\begingroup$ Each row contains around 10 attributes, so the data set is multivariate. $\endgroup$
    – sidmitra
    Nov 13, 2009 at 15:36
1
$\begingroup$

Row being a time series of observations for each video? I suggest you aggregate the cross section dimension first, to see the most general patterns like seasonality at different frequencies (daily, weekly etc). It should give you an idea of how to make the series stationary, which will allow you to use regression tools. Then a large pooled OLS regression probably with dummy variables for some of the attributes is in order, and you can proceed with classifying the residuals of this regression. It should give you a basic idea of what the data looks like, actual prediction process will be model specific and depend on which features of the data (like time series vs attributes dimension) you find the most persistent.

$\endgroup$
1
$\begingroup$

the very good book The Elements of Statistical Learning:Data Mining, Inference, and prediction is available online. It is certainly not all about statistical prediction, but it contains very interesting and non trivial chapters.

$\endgroup$
1
$\begingroup$

another good resource is: Fundamentals of Statistical Signal Processing by Kay.

It is mainly about estimation theory. Might be too academic for your needs, but it is definitely a good introduction to the theory of prediction and filtering (linear regression and Kalman filtering are also dealt with). It has strong emphasis on application and not just theory.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.